Achieving a clear visibility into the functioning of systems is of utmost importance in simplifying the management of any IT Operations environment. It is imperative to capture and analyze performance metrics in a timely manner. Sounds pretty obvious, doesn’t it ? It absolutely does!! However, it’s the follow up question of how such a capture & subsequent analysis is done, that leaves most of the IT folks in a quagmire. This is where TextFSM can help! It enables data parsing from Network Devices. With TextFSM, you can automate various scenarios especially related to Network Automation, in a production environment based on the parsed data.
If you are lucky then your operations environment may already boast of a suitable and reliable system monitoring application, one that aids you in addressing the aforementioned challenge. However, should this be not the case, you have two choices : either collaborate with your customer and procure the latest shiny tool in the market, or outline your requirements and ascertain how much of the process can be methodically automated into distinct modules. By doing so, you not only gain knowledge and expertise for yourself, but also save a substantial amount of $$$ for your customer.
While I don’t anticipate Network/System Administrators transforming into full time Developers, it’s difficult to envision an IT Operations environment where they rely solely on traditional CLI-based automation. If your environment lacks access to modern monitoring and analytical tools, and you still rely on CLI-based automation, I hope that this multi-part series will provide you with some direction on where to begin and how to optimize your automation efforts by adding a layer of visualization on top of it.
Since I am a Network/Unified Communications guy, my focus in this series will primarily revolve around Cisco CUBE Routers. The automation concepts and techniques discussed can easily be adapted to other network devices such as switches and APs, thereby enabling you to effectively capture, monitor and visualize performance metrics across your entire network.
I will start this series by covering an important tool called TextFSM that is extremely useful while working with unstructured or semi-structured data especially in the context of network automation. It is imperative that the underlying data is in a clearly formatted state before we can even begin the actual automation of use cases. In fact, the art of visualization comes even later. The table of contents is given below. You can click on the links to jump to the relevant section.
TextFSM
As per the official TextFSM Github page, TextFSM is a Python module that implements a template based state machine for parsing semi-formatted text. In other words, TextFSM allows us to capture selective information from raw text using a template file. This capability of TextFSM is particularly useful in the context of Network Automation because the CLI output of commands on Network devices result in an unstructured data. Let’s review some of the important terms from the bolded statement above.
- Template : A template is a document that describes how the input text data should be processed. It comprises of the following 3 major elements
- Values
- States
- Actions
- Semi-Formatted Text : The CLI output of a command on a network device results in a semi-formatted data. It’s categorized as semi-formatted because it’s not formatted in a way which is compatible with the interpretation capability of a computer program. In other words, it is not like JSON, XML which are the pre-dominant standards when it comes to data handling.
TextFSM Templates
A TextFSM template is the brain that decodes and captures interesting pieces of information from a semi-formatted data like a command’s output on a network device. A template is built by defining the following three elements
- Values : These are like the variables that capture the information during the parsing process. It can be considered as the column of a table. It usually follows the following syntax – Value name Regex
- Value : This is a keyword which indicates that some information needs to be recorded.
- Name : The name of the column or the variable that will store the actual parsed information.
- RegEx : The RegEx pattern that needs to be matched in the raw text input. A regex pattern should be enclosed in parenthesis – e.g (\d+) to match a pattern of 1 or more digits.
- States : After defining the values, we need to tell TextFSM about the parsing states. There are primarily 3 states – Start, End and EOF. Each state consists of one or more parsing rules written in Regex that tells TextFSM what to do when it encounters a certain type of input.
- Rules : Rules are contained within a State. The text input which in our case is the command output from CLI will be matched against each of the rules mentioned within the State. If a match happens then a certain Action will be performed. The rule starts with a caret symbol ^ and it’s followed by some text or Regex pattern. We can also reference the Value variable here using a syntax ${Value Name}.
- Actions : Once a rule is matched, an action has to be executed to complete this process. An action is identified with the ‘->’ character after the RegEx pattern in the Rule step. The type of action we will be using in this series is called “Record” action. It’s purpose is to write the output of the matching rule as a new record in the TextFSM record table.
TextFSM Template Repository
The good thing about open source development community is that you don’t always need to re-invent the wheel. There are plenty of resources already available which can be utilized to build custom solutions. The following Github link already contains a lot of pre-built TextFSM templates which you can utilize in your specific use cases
https://github.com/networktocode/ntc-templates/tree/master/ntc_templates/templates
However, it is always a good practice to think of these mediums only as stepping stones & use that understanding to build your own custom solutions. It is in this context, that we will be reviewing a very simple & straightforward example which should be enough to demonstrate the power of TextFSM templates in the context of Network Automation.
Sample Exercise
We want to extract the IP of NTP servers that the router is using for to synchronize its clock. The “show ntp associations” command on the router provides the following output
CSR1000#sh ntp associations
address ref clock st when poll reach delay offset disp
*~127.127.1.1 .LOCL. 1 6 16 377 0.000 0.000 1.204
* sys.peer, # selected, + candidate, - outlyer, x falseticker, ~ configured
CSR1000#
TextFSM Template
This template is only going to capture a value that follows a specific pattern like one or more digits.one or more digits.one or more digits.one or more digits (\d+.\d+.\d+.\d+). If you look at the command output above, the IP address also follows the same pattern, It has 8 octet values separated by a dot.
Value IP (\d+.\d+.\d+.\d+)
Start
^.*${IP} -> RecordOutput
In order to test your custom TextFSM templates, you can visit the following website
As can be seen in the following screenshot, it takes the raw text which is the output of the “show ntp associations” command and a TextFSM template you designed, as inputs and spits out the corresponding output. Click on the image to enlarge it.
I hope this tutorial was helpful in explaining the applicability of tools like TextFSM in the Network Automation world. We can not afford to jump straight into the use cases until we have a good understanding of how to structure the data. In the next post, I will explain how we can take this learning and not only solve a real world use case but also present it in an attractive visual format.
If you have any feedback/suggestions then please drop it in the comments or reach me at LinkedIn. Happy Learnings!!
Learn UC Collaboration 