| Internet-Draft | Network Working Group | July 2022 |
| Zhou, et al. | Expires 11 January 2023 | [Page] |
The Digital Twin Network is a network system with Physical Network and Twin Network, which can be mapped interactively in real time. The construction of Digital Twin Network requires real-time data of Physical Network to update the state of Twin Network. This document aims to describe the data collection requirements and provide data collection methods or tools to build the data repository for digital twin network.¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 11 January 2023.¶
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
With the deployment of Internet of Things (IoT), cloud computing and data center, etc., the scale of the current network is expanded gradually. However, the increase of network scale leads to also increasing the complexity of the current network, and it induces plenty of problems. In order to improve the autonomy ability of network and reduce potential negative effects on physical and virtual networks, we consider that an endogenous intelligent and autonomous network architecture which achieves self-optimization and decision is indispensable (in general, self-management and self-operation). The digital twin technology answers to the challenge of building self-management systems because it can optimize and validate policies through real-time and interactive mapping with physical entities.[I-D.irtf-nmrg-network-digital-twin-arch]¶
Data is the cornerstone required for constructing a digital twin for a network, namely a Digital Twin Network (DTN). In the face of large network scale, data collection, storage and management are faced with great challenges. So, data collection methods and tools should meet the requirements of target-driven, diversity, lightweight and efficiency, while being open and standardized. Among all the requirements, achieving a lightweight and efficient data collection method is of the most importance. If the full-data collection method is adopted, huge storage space and bandwidth resource is needed, especially for complex scenarios that require real-time data and traffic from multi-source and heterogeneous devices. Therefore, it is extremely important to agree on lightweight and efficient data collection, aggregation, and correlation methods, toward building the telemetry data transmission, processing, and storage required to build a DTN system.¶
PN: Physical Network¶
IMC: Instruction Management Center¶
DSC: Data Storage Center¶
DTN: Digital Twin Network¶
TSE: Telemetry Streaming Element¶
RDF: Resource Description Framework¶
CPE: Complex Event Processing¶
The monitoring data of a network is the basis to build a DTN system. Such data is collected from physical and virtual networks. It includes, but is not limited to, the following types:¶
The collection of network data for maintaining a DTN should be in target-driven and on-demand mode. It is not always necessary to collect complete network data list above because of the high cost of resources (CPU, memory, bandwidth etc.). The type, frequency and method of data collection aim to meet the application of a DTN depends on the specific network topology and application requirements.¶
The different types of network data used to maintain a DTN have several characteristics. Some data (e.g. port statistics, key link info, etc.) requires higher collecting frequency, and some data (e.g. flow status, link fault, etc.) needs to be of higher level of real-time. Some data (e.g. device status, port statistics, etc.) can be collected directly and simply via normal tools, while some data (e.g. per-flow latency, traffic matrix, etc.) can only be acquired through complex network measurement. Therefore, multiple tools or methods are needed to collect the massive data required to build the DTN entity.¶
Currently, some widely-used tools, such as SNMP, NetConf, Telemetry, INT (In-band Network Telemetry), DPI (Deep Packet Inspection), etc. can be candidate tools to collect data for digital twin network. Going forward, it is necessary to study new data collection technology in the following aspects in combination with the data requirements of network application for DTN:¶
Data collection tools and methods should be as lightweight as possible, so as to reduce the occupation of network equipment resources and ensure that data collection does not affect the normal operation of the network. The major requirements are list as below.¶
Data collection interface used to build the DTN should be open and standardized to help avoid either hardware or software vendor lock, and achieve inter-operability. The major requirements of data collection interfaces are:¶
Both raw network data and knowledge items obtained from monitoring must be able to be addressed uniquely. This means to give a unique identifier or "name" to each data or knowledge item that references it. This name will be used by caching mechanisms to store the data and provide it for clients that request it, which will also use such name.¶
The maintenance of DTN systems will not be the sole purpose of monitoring information and knowledge communication. Other applications would also request raw telemetry data or knowledge items. They can use the name to identify it. The telemetry system, following the recommendations of RFC 9232 [RFC9232], will deliver the requested data or knowledge items to the requesters as much efficiently as possible. On the one hand, items will be provided by the closest cache to the destination of the data. On the other hand, items will be replicated in the best nodes, following an efficient multi-cast spanning tree. Different underlying protocols can be used to achieve this mechanism.¶
The system that manages the DTN maps, in real time, the PN to the DTN. However the existing methods collect the full data from the PN for modeling, and do not consider problems like time-lag, insufficient storage resources, low computational efficiency and waste of bandwidth resources caused by data transmission. In order to solve these problems, this section introduces an efficient data collection method for maintaining the DTN. This data collection method is based on sending instructions to the elements of the PN for them to pre-process the data (data cleaning or knowledge representation) before sending it back to be applied to the DTN.¶
The management system structure consists of the PN and the DTN. The PN includes multiple Data Storage Centers (DSC) and Telemetry Streaming Element (TSE), and the DTN includes the Instruction Management Center (IMC) and Data Storage Center (DSC). The TSE has multiple functions, including data collection, data aggregation, data correlation, knowledge representation and query, etc. In addition, a Complex Event Processing (CEP) engine is integrated into TSE to perform queries to the streamed data. The IMC has two functions. On the one hand, it is used to manage the registration of the DSC in the PN side, and its registration information can include various key information such as the IP address of the DSC in the PN side, chosen data type, and various index names in the data, data source name and data size, etc. On the other hand, it is used to adaptively configure data collection instructions according to the collection requirements of the DSC in the DTN side and search for IP addresses to send instructions. The instruction-carrying information includes rule-based mathematical expressions, executable models in .exe format, dynamic collection frequency, parameter lists, program text files in .m format, text files with parameter configuration, and other types of files. Instructions are flexible and programmable, and can be created, modified, combined, and deleted at any time according to requirements. When the DSC of the DTN side requests data to the IMC, the IMC searches the IP address of the DSC in the database with the registration information, which is built according to critical information, such as data type and data name, and functional instructions for data processing or knowledge representation can be implemented depending on the demand configuration. The DSC of the DTN side stores the effective information after data processing and knowledge representation returned by the TSE.¶
The DSC in the PN side has two functions. On the one hand, it stores data of various types, such as performance indicators, operational status, log, traffic scheduling, business requirements, etc. On the other hand, it has the function of automatically parsing the instructions sent by the TSE. Then the operating environment of the instruction is configured according to the instruction needs, and data processing or knowledge representation is performed based on the instruction. Data processing mainly includes data cleaning, filling missing data, normalization, conflict verification, etc. Knowledge representation refers to the representation of the original data as a data structure that can be used for efficient computation. Such representation results are closer to machine language, which is conducive to the rapid and accurate construction of the model. The role of knowledge representation is to represent the original data as a data structure that can be used to efficiently calculate. Such representation results closer to the machine language, which is conducive to the rapid and accurate construction of the model.¶
+------------------------------+ +-----------------------+
| Physical Network | | Digital Twin Network |
| +-----+ +-----+ +------+ | | +------+ +-------+ |
| | | | | | | | | | | | | |
| | DSC |... | DSC | | TSE | | | | IMC | | DSC | |
| | | | | | | | | | | | | |
| +-+---+ +--+--+ +---+--+ | | +---+--+ +----+--+ |
| | | | | | | | |
+------------------------------+ +-----------------------+
| | | | |
| 1.1. Register | | |
+-----------+---------> | |
| | | | |
| | 1.2. Register | |
| +---------> | |
| | | 1.3. Register | |
| | +---------------> |
| | | 2. Data req. |
| | | <----------+
| | | 3. Query and instruction |
| | | configuration |
| | | + |
| | 4. Send instructions |
| | <---------------+ |
| | | | |
| | 5. Parse and execute | |
| | instruction | |
| 6. Data subscript. | | |
<---------------------+ | |
| 7. Knowledge | | |
| representation | | |
| 8. Data pushing | | |
+---------------------> | |
| | 9. Data aggregation and | |
| | correlation | |
| | | 10. Send processed data |
| | +-------------------------->
| | | | |
The specific process is as follows:¶
This draft describes the requirements for data collection and provides the data collection methods or tools required to build the data repository for maintaining DTN systems. These data collection methods or tools should meet the requirement of target-driven, diversity, lightweight and efficiency, while being open and standardized. Among all the requirements, lightweight and efficiency requirements are the most important. Thus, this draft provides a lightweight and efficient method for data collection that is particularly optimized for maintaining DTN systems. Going forward, more methods (transformation and aggregation functions) and tools (solutions) shall be studied to extend the contents of this draft.¶
This document has no requests to IANA.¶