The Routing Over Large Clouds Mailing List Archive by date

Cell Relay Retreat>List Archive>month:1996-Feb> msg00035



[Date Prev][Date Next][Thread Prev][Thread Next]  
  [Date Index][Thread Index][Author Index][Subject Index]

Server Synchronization revision

  • From: James Luciani <luciani@nexen.com>
  • Date: Thu, 29 Feb 1996 17:56:08 -0500

Folks,

  Included is an update of the server synchronization draft which I put out 
a few days ago. This update addresses the comments I received from Joel and 
others.  Thanks very much for the input!  The meat of the protocol is 
contained in the first 9 pages and the rest is background material and packet 
classes (which I moved to appendices).

Regards,
-- Jim Luciani
__________________________________________________________________________
James V. Luciani    Ascom Nexion                    voice: +1 508 266-3450
luciani@nexen.com   289 Great Rd., Acton MA 01720   FAX: +1 508 266-2300

-------------------------------cut here-----------------------------------







Routing Over Large Clouds Working Group                 James V. Luciani
INTERNET-DRAFT                                            (Ascom Nexion)
<draft-luciani-rolc-scsp-01.txt>                  Expires September 1996






          Server Cache Synchronization Protocol (SCSP) - NBMA


Status of this Memo

   This document is an Internet-Draft.  Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its areas,
   and its working groups.  Note that other groups may also distribute
   working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as ``work in progress.''

   To learn the current status of any Internet-Draft, please check the
   ``1id-abstracts.txt'' listing contained in the Internet-Drafts Shadow
   Directories on ds.internic.net (US East Coast), nic.nordu.net
   (Europe), ftp.isi.edu (US West Coast), or munnari.oz.au (Pacific
   Rim).

Abstract

   This document describes the Server Cache Synchronization Protocol
   (SCSP) for Non Broadcast Multiple Access (NBMA) networks.  SCSP
   attempts to solve the generalized server synchronization/cache-
   replication problem wherein a set of server entities which are bound
   to a Server Group (SG) through some means (e.g., all servers
   belonging to the same Logical IP Subnet (LIS)[1]) wish to synchronize
   the contents (or a portion thereof) of their caches.  These caches
   contain information on the state of the clients within the scope of
   interest of the SG.  An example of types of information that must be
   synchronized can be seen in NHRP using IP where the information
   includes the REGISTERED clients' IP to NBMA mappings in the SG LIS.

1. Introduction

   It is perhaps an obvious goal for any protocol to not limit itself to
   a single point of failure such as having a single server in a



Luciani                                                         [Page 1]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   client/server paradigm.  Even when there are redundant servers, there
   still remains the problem of cache synchronization; i.e.,  when one
   server becomes aware of a change in state of cache information then
   that server must propagate the knowledge of the change in state to
   all servers which are actively mirroring that state information.
   Further, this must be done in a timely fashion without putting undo
   resource strains on the servers. Assuming that the state information
   kept in the server cache is the state of clients of the server, then
   in order to minimize the burden placed upon the client it is also
   highly desirable that clients need not have complete knowledge of all
   servers which they may use.  However, any mechanism for
   synchronization should not preclude a client from having access to
   several (or all) servers.  Of course, any solution must be reasonably
   scalable, capable of using some autoconfiguration service, and lend
   itself to a wide range of authentication methodologies

   This document describes the Server Cache Synchronization Protocol
   (SCSP). SCSP solves the generalized server synchronization/cache-
   replication problem while addressing the issues described above.
   SCSP synchronizes caches (or a portion of the caches) of a set of
   server entities which are bound to a Server Group (SG) through some
   means (e.g., all NHRP servers belonging to a Logical IP Subnet
   (LIS)[1]) and may exist in an arbitrary topology as long as the
   resultant graph spans the set of servers that need to be
   synchronized.  These caches contain information on the state of the
   clients within the scope of interest of the SG.  An example of types
   of information that must be synchronized can be seen in NHRP[2] using
   IP where the information includes the REGISTERED clients' IP to NBMA
   mappings in the SG LIS.

   While SCSP is meant to solve the general server synchronization
   problem, the following sections, except where otherwise explicitly
   stated to the contrary, will draw upon the problem as it exists in
   NHRP[2].

2. Overview

   SCSP allows for an arbitrary topology as long as the resultant graph
   spans the set of servers that need to be synchronized.  SCSP borrows
   heavily from the link state protocols [3,4].  However, unlike those
   technologies, there is no costly Shortest Path First (SPF)
   calculation and there is little or no additional memory requirements
   imposed above and beyond that which is required to save the cached
   information which would exist regardless of the synchronization
   technology.

   In order to give a frame of reference for the following discussion,
   the terms Local Server (LS), Directly Connected Server (DCS), and



Luciani                                                         [Page 2]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   Remote Server (RS) are introduced.  The LS is the server under
   scrutiny; i.e., all statements are made from the perspective of the
   LS when discussing the SCSP protocol. The DCS is a server which is
   directly connected to the LS;  e.g., there exists a VC between the LS
   and DCS.  Thus, every server is a DCS with respect to every other
   server which connects to it directly, and every server is an LS which
   has zero or more DCSs directly connected to it.  An RS is a server
   that is neither an LS nor a DCS; i.e, an RS is always two or more
   hops away from an LS (whereas a DCS is always one hop away from an
   LS).

   SCSP uses three message types: "Hello", "Cache Alignment", and
   "Client State Update".  "Hello" messages are used to ascertain
   whether a DCS is operational and whether the connection between the
   LS and DCS is bidirectional, unidirectional, or non-functional.
   "Cache Alignment" (CA) messages allow an LS to synchronize its entire
   cache with that of the cache of its DCSs. "Client State Update" (CSU)
   messages are used to update the state of cache entries in servers for
   a given SG.  Sections 2.1, 2.2, and 2.3 contain a more in depth
   explanation of the Hello, CA, and CSU messages respectively.

                       +---------------+
                       |               |
              +-------@|     DOWN      |@-------+
              |        |               |        |
              |        +---------------+        |
              |            |       @            |
              |            |       |            |
              |            |       |            |
              |            |       |            |
              |            @       |            |
              |        +---------------+        |
              |        |               |        |
              |        |    WAITING    |        |
              |     +--|               |--+     |
              |     |  +---------------+  |     |
              |     |    @           @    |     |
              |     |    |           |    |     |
              |     @    |           |    @     |
            +---------------+     +---------------+
            |  BIDIRECTION  |----@|  UNIDIRECTION |
            |               |     |               |
            |  CONNECTION   |@----|  CONNECTION   |
            +---------------+     +---------------+


     Figure 1: Hello Finite State Machine (HFSM)




Luciani                                                         [Page 3]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


2.1  Hello Messages

   "Hello" messages ascertain whether a DCS is operational and whether
   the connections between the LS and DCS are bidirectional,
   unidirectional, or non-functional. Every LS MUST periodically send
   Hello messages to each of its DCSs. An LS must be configured with a
   list of DCS NBMA addresses. The mechanism for this configuration is
   beyond the scope of this document although one possible mechanisms
   would be an autoconfiguration server.

   An LS has a Hello Finite State Machine (HFSM) associated with each of
   its DCSs (see Figure 1).  The HFSM monitors the state of the
   connectivity between the LS and a particular DCS.  The HFSM starts in
   the "Down" State and transitions to the "Waiting" State after NBMA
   level connectivity has been established.  Once in the Waiting State,
   the LS starts sending Hello messages to the DCS.  The Hello message
   includes: a Sender ID which is set to the LS's ID (LSID), a Receiver
   ID which identifies the expected receiver of the Hello message and is
   initially set to zero, and a HelloInterval and DeadFactor which will
   be described below.  At this point, The DCS may or may not be sending
   its own Hello messages to the LS.  In either case, upon receiving the
   LS's Hello, the DCS copies the LSID from the Sender ID (SID) field of
   the LS's Hello message.  When the DCS sends its next Hello to that
   LS, the DCS includes the LSID in the Receiver ID field and its own ID
   (the DCSID) in the Sender ID field. When the LS receives the DCS's
   Hello message, it will know that the DCS has received the LS's Hello
   message and thus bidirectional communication is possible at which
   point the HFSM transitions from the Waiting State to the
   "Bidirectional Connection" State.

   If an LS which is not in the down state receives a Hello message from
   a DCS and that message has a zero in the Receiver ID field then the
   HFSM for that DCS transitions to the "Unidirectional Connection"
   State.  If while in the Unidirectional State, the LS receives a
   subsequent Hello message from that DCS and that message contains a
   Receiver ID equal to the LSID then the HFSM transitions to the
   Bidirectional Connection State.  Any abnormal event, such as
   receiving a malformed Hello message or receiving a Hello message with
   a Receiver ID which is neither the LSID nor zero, causes the HFSM to
   transition to the Waiting State; however, a loss of NBMA connectivity
   causes the HFSM to transition to the Down State.

   Hello messages also contain a HelloInterval and a DeadFactor.  The
   Hello interval advertises the time between sending of consecutive
   Hello messages by a server.  That is, if the time between reception
   of Hello messages from a DCS exceeds the HelloInterval advertised by
   that DCS then the next Hello message is to be considered late by the
   LS.  If the LS does not receive a Hello message within the interval



Luciani                                                         [Page 4]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   HelloInterval*DeadFactor seconds then the LS MUST consider the DCS to
   be stalled at which point the LS should transition the HFSM to the
   Waiting State.  A DeadFactor is also advertised on a per server
   basis.


                   +------------+
                   |            |
              +---@|    DOWN    |
              |    |            |
              |    +------------+
              |          |
              |          |
              |          @
              |    +------------+
              |    |Master/Slave|
              |    |            |@---+
              |    |Negotiation |    |
              |    +------------+    |
              |          |           |
              |          |           |
              |          @           |
              |    +------------+    |
              |    |   Cache    |    |
              |    |            |    |
              |    | Summarize  |    |
              |    +------------+    |
              |          |           |
              |          |           |
              |          @           |
              |    +------------+    |
              |    |   Update   |    |
              |    |            |    |
              |    |   Cache    |    |
              |    +------------+    |
              |          |           |
              |          |           |
              |          @           |
              |    +------------+    |
              |    |            |    |
              +----|  Aligned   |----+
                   |            |
                   +------------+


     Figure 2: Cache Alignment Finite State Machine





Luciani                                                         [Page 5]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


2.2 Cache Alignment Messages

   "Cache Alignment" (CA) messages allow an LS to synchronize its entire
   cache with that of the cache of its DCSs.  That is, CA messages allow
   a booting LS to synchronize with its DCSs.  A CA message contains a
   CA header followed by zero or more Client State Advertisement Summary
   records (CSAS records).

   An LS has a Cache Alignment Finite State Machine (CAFSM) associated
   (see Figure 2) with each of its DCSs.  The CAFSM starts in the Down
   State.  When the HFSM reaches the Bidirectional State, the CAFSM
   transitions to the Master/Slave Negotiation State.  The Master/Slave
   Negotiation State causes either the LS or DCS to take on the role of
   master over the cache alignment process.

   When the LS's CAFSM reaches the Master/Slave Negotiation State, the
   LS will send a CA message to the DCS associated with the CAFSM.  The
   first CA message which the LS sends includes no CSAS records and a CA
   header which contains the LSID in the Sender ID field, the DCSID in
   the Receiver ID field, a sequence number, and three bits.  These
   three bits are the M (Master/Slave) bit, the I (Initialization of
   master) bit, and the O (More) bit. In the first CA message sent by
   the LS to a particular DCS, the M, O, and I bits are set to one.  If
   the LS does not receive a CA message from the DCS in CAReXmtInterval
   seconds then it resends the CA message it just sent.  The LS
   continues to do this until the CAFSM transitions to the Cache
   Summarize State or until the HFSM transitions out of the
   Bidirectional State.  Any time the HFSM transitions out of the
   Bidirectional State, the CAFSM transitions to the Down State.

   When the LS receives a CA message from the DCS while in the
   Master/Slave Negotiation State, the role the LS plays in the exchange
   depends on packet processing as follows:

   1) If the CA from the DCS has the M, I, and O bits set to one and there are no
      CSAS records in the CA message and the SenderID as specified in the DCS's CA
      is larger than the LSID then
     a) The timer counting down the CAReXmtInterval is stopped.
     b) The CAFSM corresponding to that DCS transitions to the Cache Summarize State
        and the LS takes on the role of slave.
     c) The LS adopts the sequence number it received in the CA message as its own
        sequence number.
     d) The LS sends a CA message to the DCS which is formated as follows:
        the M and I bits are set to zero, the Sender ID field is set to the LSID,
        the Receiver ID field is set to the DCSID, and the sequence number is set
        to the sequence number that appeared in the DCS's CA message.
        If there are CSAS records to be sent (i.e., if the LS's cache
        is not empty) then the O bit is set to one and the initial set of CSAS



Luciani                                                         [Page 6]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


        records are included in the CA message.

   2) If the CA message from the DCS has the M and I bits off and the Sender ID as
      specified in the DCS's CA message is smaller than the LSID then
     a) The timer counting down the CAReXmtInterval is stopped.
     b) The CAFSM corresponding to that DCS transitions to the Cache Summarize State
        and the LS takes on the role of master.
     c) The LS must process any CSAS records in the received CA.
        An explanation of record processing is given below.
     d) The LS sends a CA message to the DCS which is formated as follows:
        the M bit is set to one, I bit is set to zero, the Sender ID
        field is set to the LSID, the Receiver ID field is set to the DCSID, and
        the LS's current sequence number is incremented by one and placed in
        the CA message.   If there are any CSAS records to be sent from the LS to
        the DCS (i.e., if the LS's cache is not empty) then the O bit is set to one
        and the initial set of CSAS records are included in the CA message that the
        LS is sending to the DCS.

   3) Otherwise, the packet must be ignored.

   At any given time, the master or slave have at most one outstanding
   CA message.  Once the LS's CAFSM has transitioned to the Cache
   Summarize State the sequence of exchanges of CA messages occurs as
   follows.

   1) If the LS receives a CA message with the M bit set incorrectly
      (e.g., the M bit is set in the CA of the DCS and the LS is master)
      or if the I bit is set then the CAFSM transitions back to the
      Master/Slave Negotiation State.

   2) If the LS is master and the LS receives a CA message with a sequence
      number which is one less than the LS's current sequence number then
      the message is a duplicate and the message MUST be discarded.

   3) If the LS is master and the LS receives a CA message with a sequence
      number which is equal to the LS's current sequence number then the
      CA message MUST be processed.  An explanation of message processing
      is given below.  As a result of having received the CA message from
      the DCS the following will occur:
     a) The timer counting down the CAReXmtInterval is stopped.
     b) The LS must process any CSAS records in the received CA message.
     c) Increment the LS's sequence number by one.
     d) The cache exchange continues as follows:
       1) If the LS has no more CSAS records to send and the received CA
          message has the O bit off then the CAFSM transitions to the Update
          Cache State.
       2) If the LS has no more CSAS records to send and the received CA
          message has the O bit on then the LS sends back a CA message



Luciani                                                         [Page 7]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


          (with new sequence number) which contains no CSAS records and
          with the O bit off.  Reset the timer counting down the
          CAReXmtInterval.
       3) If the LS has more CSAS records to send then the LS sends the next
          CA message with the LS's next set of CSAS records.  If LS is sending
          its last set of CSAS records then the O bit is set off otherwise the
          O bit is set on. Reset the timer counting down the CAReXmtInterval.

   4) If the LS is slave and the LS receives a CA message with a sequence
      number which is equal to the LS's current sequence number then the
      CA message is a duplicate and the LS MUST resend the CA message
      which it had just sent to the DCS.

   5) If the LS is slave and the LS receives a CA message with a sequence
      number which is one more than the LS's current sequence number then
      the message is valid and MUST be processed.  An explanation of message
      processing is given below.  As a result of having received the CA
      message from the DCS the following will occur:

     a) The LS must process any CSAS records in the received CA message.
     b) Set the LS's sequence number to the sequence number in the CA
        message.
     c) The cache exchange continues as follows:
       1) If the LS had just sent a CA message with the O bit off and the
          received CA message has the O bit off then the CAFSM transitions to
          the Update Cache State and the LS sends a CA message with no CSAS
          records and with the O bit off.
       2) If the LS still has CSAS records to send then the LS MUST send
          a CA message with CSAS records in it.  If the message being sent
          from the LS to the DCS contains the last CSAS records that the
          LS needs to send then the CA is sent with the O bit off.

   6) If the LS is slave and the LS receives a CA message with a sequence
      that is neither equal to or one more than the current LS's sequence
      number then an error has occurred and the CAFSM transitions to the
      Master/Slave Negotiation State.

   CA message processing occurs as follows:

   The LS makes a list of those cache entries which are more "up to
   date" in the DCS than the LS's own cache.  A CSA record is more "up
   to date" than the corresponding cache entry in the LS if
     1) the sequence number in the CSA record is "larger" (based on the lollipop
        method) than that found in the LS's corresponding cache entry
     2) the combination of the Client ID and CSA Originator ID do not exist in the
        LS's cache
   During this process, the DCS makes a similar list with respect to the
   LS.  The previously mentioned list is called the CSA Request List



Luciani                                                         [Page 8]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   (CRL).  If the CRL of the LS is empty upon transition into the Update
   Cache State then the CAFSM immediately transitions into the Aligned
   State. If the CRL is not empty then the LS solicits the relevant CSA
   records from the DCS associated with the CAFSM and when the LS has
   all the updated CSA record information it transitions into the
   Aligned State.  The LS solicits the relevant CSA records by forming
   CSU Solicit (CSUS) messages from the CRL. CSUS messages contain a
   CSUS header and CSAS records from the CRL.  The LS then sends the
   CSUS messages to the DCS. The DCS responds to the CSUS messages by
   sending CSU messages containing the appropriate CSA records to the
   LS.  The DCS acts in a similar manner as does the LS with respect to
   acquiring updated CSA records for the CSAS records in the CRL.  In
   this way, both LS and DCS databases are synchronized.  At most one
   CSUS message will be outstanding at any given time.

   Just before the first CSUS message is sent from an LS to the DCS
   associated with the CAFSM, a timer is set to CSUSReXmtInterval
   seconds.  If all the CSA records corresponding to the CSAS records in
   the CSUS message have not been received by the time that the timer
   expires then a new CSUS message will be created which includes all
   the outstanding CSA records plus additional CSAS records not covered
   in the previous CSUS message.  The new CSUS message is then sent to
   the DCS. If, at some point before the timer expires, all CSA record
   updates have been received for all the CSAS records included in the
   previously sent CSUS message then the timer is stopped and if there
   are additional CSAS records that were not covered in the previous
   CSUS message but were in the CRL then the timer is reset and a new
   CSUS message is created which contains CSAS records from the CRL
   which have not yet been sent to the DCS.  This process continues
   until all the CSAS records that were in the CRL have been updated in
   the LS.  When the LS has a completely updated cache then the LS's
   CAFSM transitions to the Aligned State as previously mentioned.

2.3. Client State Update Messages

   "Client State Update" (CSU) messages are used to update the state of
   cache entries in servers for a given SG.  An LS may send/receive a
   CSU to/from a DCS only when the corresponding CAFSM is in either the
   Aligned State or the Update Cache State.

   A CSU message is sent from an LS to each of its DCSs when the LS
   observes changes in the state of one or more clients in the SG. The
   change in state of a particular client is noted in a CSU message via
   a "Client State Advertisement" (CSA) record within the CSU.  In this
   way, state changes are propagated throughout the SG.

   Examples of such changes in state are as follows:




Luciani                                                         [Page 9]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


       1) an LS receives a request to add an entry to its cache
          (e.g., NHRP Registration Request or an administrative
          intervention),

       2) an LS receives a request to remove an entry from its cache
          (e.g., NHRP Purge Request or administrative intervention),

       3) a cache entry has timed out in the LS's cache, has been refreshed
          in the LS's cache, or has been administratively modified
          (e.g., in NHRP, an Internetworking address to NBMA address binding
          has timed out or has been refreshed).

   After receiving a CSU, an LS acknowledges it by sending a CSU Reply.
   Each CSA which the LS has not already seen is propagated to each of
   its the DCS's except the DCS from which LS originally received the
   CSU.

   A LS responds to CSUS messages by sending CSU messages containing the
   appropriate CSA records to the DCS.  The LS acknowledges the CSU
   messages as described above.

Conclusions

   While the above text is couched in terms of synchronizing the
   knowledge of the state of a client within the cache of servers
   contained in a server grouping, this solution generalizes easily to
   any number of database synchronization problems (e.g., LECS
   synchronization).  In such a case, the Client ID (CID) and Client
   state would be replaced by a unique token and an octet string
   describing the database entry being synchronized. The appendices
   below show how SCSP can be implemented in terms of packet syntax
   specific to a set of other protocols.

Appendix A:  Terminology

  This appendix introduces the terminology associated with SCSP.

A.1 Abbreviations

   CA - Cache Alignment Message

   CAFSM - Cache Alignment Finite State Machine

   CID - Client ID

   CRL - CSA Request List

   CSA - Client State Advertisement



Luciani                                                        [Page 10]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   CSAS - Client State Advertisement Summary

   CSU - Client State Update

   CSUS - Client State Update Solicit

   DCS - Directly Connected Server

   HFSM - Hello Finite State Machine

   I - Initialize bit

   LS - Local Server

   LSID - Local Server ID

   M - Master/Slave bit

   O - More bit

   RS - Remote Server

   SG - Server Group

   SID - Server ID

A.2 Definitions

   Cache Alignment message (CA message)
     These messages allow an LS to synchronize its entire cache
     with that of the cache of one of its DCSs.

   Cache Alignment Finite State Machine (CAFSM)
     The CAFSM monitors the state of the cache alignment between an LS
     and a particular DCS.  There exists one CAFSM per DCS as seen from
     an LS.

   Client ID (CID)
     The CID is an unique token which identifies a client whose state
     is being kept in a server's cache.  This value
     might be taken from the protocol address of the client.

   CSA Request List (CRL)
     When CA messages are exchanged between an LS and one of its DCSs, the LS
     makes a list of those cache entries which are more recent in the DCS (based
     on a CSAS sequence number) than the LS's own entry and adds to that list any
     entry in the DCS which is not already in its cache. This list is the CRL.




Luciani                                                        [Page 11]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   Client State Advertisement record (CSA record)
     A CSA is a record within a CSU message which identifies an update
     to the status of a "particular" client.

   Client State Advertisement Summary record (CSAS record)
     A CSAS contains a summary of the information in a CSA.  A server will send
     CSAS records describing its cache entries to another server
     during the cache alignment process.  CSAS records are also included
     in a CSUS messages when an LS wants to request the entire CSA from
     the DCS.  The LS is requesting the CSA from the DCS because the LS
     believes that the DCS has a more recent view of the state of the
     cache entry in question.

   Client State Update message (CSU message)
     This is a message sent from an LS to its DCSs when the LS
     becomes aware of a change in state of a client.

   Client State Update Solicit message (CSUS message)
     This message is sent by an LS to its DCS after the LS and DCS
     have exchanged CA messages.   The CSUS message contains one or more
     CSAS records which represent solicitations for entire CSA records
     (as opposed to just the summary information held in the CSAS).

   Directly Connected Server (DCS)
     The DCS is a server which is directly connected to the LS;
     e.g., there exists a VC between the LS and DCS.
     This term, along with the terms LS and RS, is used to give a frame
     of reference when talking about servers and their synchronization.
     Unless explicitly stated to the contrary, there is no implied
     difference in functionality between a DCS, LS, and RS.

   Hello Finite State Machine (HFSM)
     An LS has a HFSM associated with each of its DCSs.  The HFSM monitors
     the state of the connectivity between the LS and a particular DCS.

   Initialize bit (I bit)
     This bit is included in a CA message.  When set, this bit indicates
     that the sender of the CA wishes to negotiate for Master/Slave server
     status in the cache alignment process.

   Local Server (LS)
     The LS is the server under scrutiny; i.e., all statements are made
     from the perspective of the LS.
     This term, along with the terms DCS and RS, is used to give a frame
     of reference when talking about servers and their synchronization.
     Unless explicitly stated to the contrary, there is no implied
     difference in functionality between a DCS, LS, and RS.




Luciani                                                        [Page 12]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   Local Server ID (LSID)
     The LSID is a unique token that identifies an LS.  This value
     might be taken from the protocol address of the LS.

   Master/Slave bit (M bit)
     This bit is included in a CA message.  When set, this bit indicates
     that the sender of the CA wishes to be Master of the cache alignment
     process.

   More bit (O bit)
     This bit is included in a CA message.  When set, this bit indicates
     that the sender of the CA has more CA messages to send above and
     beyond the message it is currently sending.

   Remote Server (RS)
     An RS is a server that is neither an LS nor a DCS and unless otherwise
     stated an RS refers to a server in the SG.
     This term, along with the terms LS and DCS, is used to give a frame
     of reference when talking about servers and their synchronization.
     Unless explicitly stated to the contrary, there is no implied
     difference in functionality between a DCS, LS, and RS.

   Server Group (SG)
     The SCSP synchronizes caches (or a portion of the caches) of a set
     of server entities which are bound to a SG through some means
     (e.g., all servers belonging to a Logical IP Subnet (LIS)[1]).  Thus
     an SG is just a grouping of servers around some commonality.

   Server ID (SID)
     The SID is a unique token that identifies a given server.  This value
     might be taken from the protocol address of the server.

Appendix B:  Packet Formats

   This appendix includes packet formats for various protocols which
   would benefit from server synchronization.  This section is still
   under construction.

B.1  Packet Formats for NHRP

   The packet formats shown below show packet types to be added to the
   NHRP protocol in order to support SCSP.  The terms are not exactly
   the same since I am attempting to map SCSP into a "legacy" protocol
   and thus the terms used should be those of the legacy protocol and
   not those of SCSP.  However, I have attempted to point out
   equivalences between terms wherever possible.





Luciani                                                        [Page 13]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


B.1.1 NHRP Cache Alignment (NHRP CA)

   The Cache Alignment (CA) message allows an LS to synchronize its
   entire cache with that of the cache of its DCSs.  The NHRP CA message
   has a type code of 11.  The Mandatory Part has the following format:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Sender ID Len | Recvr ID Len  |M|I|O|  unused |  No. of CSASs |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   CA  Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Sender ID (variable length)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                Receiver ID (variable length)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        CSAS Record                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                              .......
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        CSAS Record                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Sender ID Len
     This field holds the length in octets of the Sender ID.

   Recvr ID Len
     This field holds the length in octets of the Receiver ID.

   M
     This bit is part of the negotiation process for the cache
     alignment.  When this bit is set then the sender of the CA message
     is indicating that it wishes to lead the alignment process.  This
     bit is the "Master/Slave bit".

   I
     When set, this bit indicates that the sender of the CA message
     believes that it is in a state where it is negotiating for the
     status of master or slave.  This bit is the "Initialization bit".

   O
     This bit indicates that the sender of the CA message has more CSAS
     records to send.  This implies that the cache alignment process
     must continue.  This bit is the "More bit" despite its dubious
     name.

   No. of CSASs



Luciani                                                        [Page 14]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


     This field contains the number of Client State Advertisements
     Summaries (CSASs) contained in the NHRP CA message.

   CA Sequence Number
     A value which provides a unique identifier to aid in the sequencing
     of the cache alignment process.  The slave NHS always copies the
     sequence number from the master NHS's previous NHRP CA message into
     its current NHRP CA message thus acknowledging the master's CA
     message.  When the slave receives a "higher" sequence number then
     the number that the slave previously sent then the slave's previous
     NHRP CA message is acknowledged.  A "larger" sequence number means
     a more recent CA message. Larger is defined in the well known
     lollipop fashion.

   Sender ID
     This is the protocol address of the NHS which is sending the NHRP
     CA message.

   Receiver ID
     This is the protocol address of the NHS which is to receive the
     NHRP CA message.

   CSAS record
     See Section B.1.1.1.

B.1.1.1 Client State Advertisement Summary Record (CSAS record):

   The client state advertisement summary is a summarization of the CSA.
   A CSAS contains the following:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Client ID Len  |CSA Orig ID Len|            unused             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    CSA Sequence Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Client Protocol Address (variable length)            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     CSA Originator Protocol Address (variable length)         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Client ID Len
     This field holds the length in octets of the Client ID.

   CSA Orig ID Len
     This field holds the length in octets of the CSA Originator ID.




Luciani                                                        [Page 15]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   CSA Sequence Number
     This field contains a sequence number that identifies the CSA
     record instance for the given client.  A "larger" sequence number
     means a more recent advertisement. Larger is defined in the well
     known lollipop fashion.

   Client ID
     This field identifies the protocol address of the client whose
     state is being kept in the NHSs' cache.

   CSA Originator ID
     This field contains the protocol address of the NHS which
     originated the CSA record.

B.1.2 NHRP Client State Update Request (NHRP CSU Request)

   The NHRP Client State Update Request (CSU Request) message is used to
   update the state of cache entries in NHSs which are attached to the
   NHS sending the message.   A NHRP CSU Request message is sent from
   one NHS (the LS) to another directly connected NHS (the DCS) when the
   LS observes changes in the state of one or more clients.  This
   observation may be a result of receiving a CSU from another DCS or as
   a result of some event occurring for a client that has registered
   with it.  The change in state of a "particular" client is noted in a
   CSU message via a "Client State Advertisement" (CSA) record within
   the CSU.  The NHRP CSU Request message has a type code of 12. The
   Mandatory Part has the following format:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Sender ID Len | Recvr ID Len  |    unused     |  No. of CSAs  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                   CSU Sequence Number                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Sender ID (variable length)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                Receiver ID (variable length)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         CSA Record                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                              .......
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                         CSA Record                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Sender ID Len
     This field holds the length in octets of the Sender ID.



Luciani                                                        [Page 16]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   Recvr ID Len
     This field holds the length in octets of the Receiver ID.

   No. of CSAs
     This field contains the number of Client State Advertisements
     (CSAs) contained in the NHRP CSU message.

   CSU Sequence Number
     A value which, when coupled with the address of the source,
     provides a unique identifier for the NHRP CSU Request This value is
     equivalent to the CSU Sequence Number in SCSP. A "larger" sequence
     number means a more recent advertisement. Larger is defined in the
     well known lollipop fashion.

   Sender ID
     This is the protocol address of the NHS which is sending the NHRP
     CSU message.

   Receiver ID
     This is the protocol address of the NHS which is to receive the
     NHRP CSU message.

   CSA Record
       See Section B.1.2.1.

B.1.2.1 CSA Record

   The Client State Advertisement (CSA) record contains the information
   necessary to relate the current state of a client to the NHSs being
   synchronized.  There are zero or more NHRP CSA records in an NHRP CSU
   Request message.  The contents of a record is as follows:




















Luciani                                                        [Page 17]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Client ID Len  |CSA Orig ID Len|            unused             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    CSA Sequence Number                        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |            Client NBMA Address (variable length)              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |          Client NBMA Subaddress (variable length)             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Client ID (variable length)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     CSA Originator Protocol Address (variable length)         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               Client State (variable length)                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Client ID Len
     This field holds the length in octets of the Client ID.

   CSA Orig ID Len
     This field holds the length in octets of the CSA Originator ID.

   CSA Sequence Number
     This field contains a sequence number that identifies the CSA
     record instance for the given client.  A "larger" sequence number
     means a more recent advertisement. Larger is defined in the well
     known lollipop fashion.

   Client NBMA Address
     The Source NBMA address field is the address of the source station
     which is sending the Next Hop Resolution Request. If the field's
     length as specified in ar$shtl is 0 then no storage is allocated
     for this address at all.

   Client NBMA SubAddress
     The Source NBMA subaddress field is the address of the source
     station which is sending the Next Hop Resolution Request.  If the
     field's length as specified in ar$sstl is 0 then no storage is
     allocated for this address at all.

   Client ID
     This field identifies the protocol address of the client whose
     state is being kept in the NHSs' cache.

   CSA Originator ID
     This field contains the protocol address of the NHS which



Luciani                                                        [Page 18]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


     originated the CSA record.

   Client State

     This field/record contains an octet string which identifies the
     current state of the client.  The field/record is broken down as
     follows:

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |             State             |  Maximum Transmission Unit    |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |          Holding Time         |  Other Data (variable length) |
     +-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+

     State
       This field contains a value which represents the change in state
       of the client.  For example:

         0 - Client is registered and available.

         1 - Holding timer expired for client.

         2 - Client reregistered.

         3 - Client has been purged.

     Maximum Transmission Unit
       This field represents the acceptable Maximum MTU for connections
       to the client.

     Holding Time
       The Holding Time field specifies the number of seconds for which
       the client information specified is valid.  Cached information
       SHALL be discarded when the holding time expires.

     Other Data
       This is a variable length octet string which is potentially
       vendor specific.  This may be encoded in a way similar to the
       Vendor Private extension.

B.1.3 NHRP Client State Update Reply (NHRP CSU Reply)

   The NHRP Client State Update Reply (CSU Reply) message is used to
   acknowledge the reception of NHRP Client State Update Request.  A
   NHRP CSU Reply message is sent from one NHS (the DCS) to the NHS (the
   LS) which sent the original NHRP CSU Request.  The NHRP CSU Reply



Luciani                                                        [Page 19]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


   message has a type code of 12. The Mandatory Part of the NHRP CSU
   Reply is the same as that of the NHRP CSU Request so that when an NHS
   receives an NHRP CSU Request all that needs to be done to reply to it
   is to change the type code to 13 and send the packet back.  However,
   the Mandatory part may be truncated from the NHRP CSU Request at (but
   not including) the Client State field.

B.1.4 NHRP Client State Update Solicit Message (NHRP CSUS message)

   The NHRP CSUS message contains a CSU header and zero or more CSAS
   records. This message allows one NHS (LS) to solicit the entirety of
   CSA data stored in the cache of a directly connected NHS (DCS).  The
   DCS responds with CSU messages containing the appropriate CSAs.  The
   NHRP CSUS message has a type code of 14.  The Mandatory Part has the
   same format as the CA message; however the M, I, and O bits are not
   meaningful in this context and are set to zero.  Also, the CA
   Sequence becomes the CSUS Sequence Number and CSUS Sequence number is
   from a different numbering space than the CA Sequence number.

B.1.5 NHRP Hello:

   The NHRP Hello Request message is used to check connectivity between
   the sending NHS (the LS) and one of its directly connected neighbor
   NHSs (the DCSs). The NHRP Hello Request packet has a type code of 15.
   The Mandatory Part has the following format:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Sender ID Len | Recvr ID Len  |           unused              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         HelloInterval         |          DeadFactor           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                    Sender ID (variable length)                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                  Receiver ID (variable length)                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Sender ID Len
     This field holds the length in octets of the Sender ID.

   Recvr ID Len
     This field holds the length in octets of the Receiver ID.

   HelloInterval
     The hello interval advertises the time between sending of
     consecutive Hello Packets by an LS.  If the time between Hello
     packets exceeds the HelloInterval then the Hello is to be



Luciani                                                        [Page 20]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


     considered late by the DCS.  On the other hand, if the LS does not
     receive a Hello Reply within its HelloInterval then the LS resends
     the same Hello message it sent previously

   DeadFactor
     This is a multiplier to the HelloInterval. If a DCS does not
     receive a Hello packet within the interval HelloInterval*DeadFactor
     from an LS that advertised the HelloInterval then the DCS MUST
     consider the LS to be stalled at which point the DCS should
     transition to the Waiting State.   On the other hand, if the LS
     does not receive a Hello Reply within DeadFactor*HelloInterval then
     one of two things happens: 1) if the LS has received Hello messages
     from the DCS during this time then the LS transitions to the
     Unidirectional State; otherwise, 2) the LS transitions to the
     Waiting State.

   Sender ID
     This is the protocol address of the NHS which is sending the NHRP
     Hello Request.

   Receiver ID
     This is the protocol address of the NHS which is to Reply to the
     NHRP Hello Request.  If the sender does not know this address then
     the sender sets it to zero and it will be filled in the subsequent
     reply.

B.2:  Packet Formats For ATMARP

      Work in progress.

B.3:  Packet Formats For IPMC

      Work in progress.

B.4:  Packet Formats For LECS

      Work in progress.


References

[1] "Classical IP and ARP over ATM", Laubach, RFC 1577.

[2] "NBMA Next Hop Resolution Protocol (NHRP)", Katz, Piscitello, Cole,
    Luciani, draft-ietf-rolc-nhrp-07.txt.

[3] "OSPF Version 2", Moy, RFC1583.




Luciani                                                        [Page 21]

INTERNET-DRAFT                 SCSP-NBMA          Expires September 1996


[4] "PNNI Draft Specification", Dykeman, Goguen, ATM Forum 94-0471R15
    (Straw Vote), 1996.


Acknowledgments

   This I-D is a distillation of issues raised during private
   discussions, on the IP-ATM mailing list, and during the Dallas IETF
   (12/95). Thanks to all who have contributed but particular thanks to
   Andy Malis, Raj Nair, and Matthew Doar of Ascom Nexion.  I would also
   like to thank Joel Halpern of Newbridge for comments that lead to a
   tighter document.

Author's Address

   James V. Luciani
   Ascom Nexion
   289 Great Road
   Acton, MA 01720-4379 USA

   Phone:  +1 508 266 3450
   Email: luciani@nexen.com





























Luciani                                                        [Page 22]