Tag Data Retention

This feature enables reader to buffer the tag events and stream data back to server in case of network issues or server failures.

• Current retention configuration can be queried using GET /config API for Local REST deployment or get_config command for MQTT deployments. Note that retention is enabled by default for Tag Data Interface.

• Sample retention configuration:

  {
      "maxEventRetentionTimeInMin": 30,
      "maxNumEvents": 150000,
      "throttle": 100
  }
  

  • maxEventRetentionTimeInMin: The maximum time window before reconnect for which the tag data events are retained in case of a network disconnect. Messages that arrived prior to this time are discarded.

  • maxNumEvents: Maximum number of events to be retained. (default: 150000 Events). Tags that exceed the maxNumEvents are discarded.

  • throttle: Rate to which reader should send retained events to server. (default: 500 Tags/second)

    • For large number of reader deployments it is recommended to adjust the throttle depending on the number of readers deployed, tag read rate and server processing capabilities.

    • Throttle value should be greater than the read rate to reduce bottlenecks.

• The total tags retained will be determined by which ever configuration hits first maxEventRetentionTimeInMin or maxNumEvents.

• Retention configuration can be updated using PUT /config API for Local REST deployment or set_config command for MQTT deployments.

• Retention can be disabled by configuring the maxNumEvents to 0.

• Retention can also be configured to retain events forever up to the maxNumEvents by setting maxEventRetentionTimeInMin to 0.

Note

Note that retained messages are lost if the reader is rebooted.

TCPIP Endpoints

client disconnection detection

For retention in the TCPIP endpoint, apart from the retention parameters (maxEventRetentionTimeInMin, maxNumEvents, and throttle), the following parameters are also significant:

• keepAlive: The time (in seconds) the connection needs to remain idle before TCP begins sending keepalive probes to check if the connection is still alive. The minimum value for the keepAlive is 60 seconds, the maximum is 14400 seconds, and the default is 7200 seconds.

• KeepAlive Interval: The interval (in seconds) between successive keepalive probes if no acknowledgment is received from the remote host. The minimum value for the keepalive interval is 60 seconds, the maximum is 120 seconds, and the default is 60 seconds.

• KeepAlive Count: The maximum number of keepalive probes TCP should send before deciding that the connection is dead and closing it. The minimum value for the KeepAlive Count is 1, the maximum is 20, and the default is 10.

• Sample keepAlive configuration:

  {
      "keepAlive": 7200,
      "keepAliveCount": 10,
      "keepAliveInterval": 60
  }
  

The total keepalive duration is calculated using the expression:

total keepalive duration = keepalive + (keepalive interval * keepalive count)

For instance, if the keepalive is set to 540 seconds, the keepalive interval is 60 seconds, and the keepalive count is 1, then the calculation 540 + (60 * 1) results in a total of 600 seconds which keepAlive is set 10 minutes.

The minimum supported keepalive duration is 2 minutes

keepalive configuration and data retention behaviour

The tag data retention behaviour as seen by the TCP client depends on the keepalive configuration and the total duration of network disconnection or client graceful shudown. Tag data retention on the reader is applied only after the a disconnection is detect by the reader. Hence it may be possible that in a case where the network disconnect duration is shorter than the keepalive duration tags data may still be retained even if retention configuration is disabled. This is explained in section Disconnection duration shorter than keep alive duration. In cases where tag data retention is not desired it is recommended to set the keep alive configuration to a minimum value possible which is 2 minutes. this is explained in keepAlive calculation section.

keepalive configuration and tag data behaviour when tag datra retention is enabled

• Scenario 1: Disconnection duration shorter than keep alive duration:

  • Reader retains the tag events for the duration of the network disconnect.

  • On reconnect, reader sends the retained events to the client.

• Scenario 2: Disconnection duration longer than keep alive duration:

  • Reader retains the tag events for the duration of the network disconnect.

  • On reconnect, reader sends the retained events to the client.

  • Some tags that are read before the keep alive duration may be lost.

• Scenario 3: client graceful disconnection:

  • Reader retains the tag events for the duration of the network disconnect.

  • On reconnect, reader sends the retained events to the client.

keepalive configuration and tag data behaviour when tag data retention is disabled

• Scenario 1: Disconnection duration shorter than keep alive duration:

  • Reader retains the tag events for the duration of the network disconnect.

  • On reconnect, reader sends the retained events to the client.

• Scenario 2: Disconnection duration longer than keep alive duration:

  • The reader will not retain tag events during the period of network disconnection.

  • on reconnect, reader sends the tag events that arrived after the network reconnect.

• Scenario 3: client graceful disconnection:

  • The reader will not retain tag events during the period of network disconnection.

  • on reconnect, reader sends the tag events that arrived after the network reconnect.

Sample TCPIP Client

import socket
import time

def connect_to_rfid_reader(host, port):
    try:
        client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        client_socket.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
        client_socket.connect((host, port))
        print(f"Connected to RFID reader at {host}:{port}")
        command = "sending query to server"
        client_socket.sendall(command.encode('utf-8'))

        while True:
            try:
                response = client_socket.recv(4096)
                if not response:
                    print("No data received, connection might be closed by server.")
                    break

                print("Received response from RFID reader:", response.decode('utf-8'))
            
            except socket.error as e:
                print(f"Socket error during data reception: {e}")
                break
        
    except socket.error as e:
        print(f"Socket error during connection: {e}")
        print("Attempting to reconnect in 10 seconds...")
        time.sleep(10)
    
    finally:
        client_socket.close()
        print("Connection closed, retrying...")

if __name__ == "__main__":
    rfid_reader_ip = "169.254.175.83"
    rfid_reader_port = 8081

    connect_to_rfid_reader(rfid_reader_ip, rfid_reader_port)

Note

If the network reconnection time exceeds the keepalive period, the reader will close the connection. To resume receiving tags, a new connection must be established, its recomended to have keepalive implementation in client side so that it will automatically close the connection when it cannot reach the Reader.

MQTT Endpoints

Disconnect Detection

MQTT disconnect is detected based on the keepalive interval set in the MQTT Endpoint. The keepalive interval is the maximum time interval between messages sent or received by the client. If the keepalive interval is exceeded, the client is disconnected by the server. The keepalive interval is set in seconds and is used to ensure that the client is still connected.

Important

Data Drops: In scenarios where there is a heavy flow of data events, users may notice data drops. Setting the MQTT QoS level to 2 can guarantee that data is not lost in transit at the cost of higher resource utilization.

Important

Data Duplication: In scenarios where network connection is lost, on reconnection there is a chance the client may receive duplicate data events. These events can be identified and filtered out based on their event numbers.