Loading data into PathWave Manufacturing Analytics (Part 2)
Overview
In this part two(2) of Loading Data into PathWave Manufacturing Analytics (PMA), we will continue to look into how data parser deal with some common use cases in manufacturing. As the number of test devices being added into a production line increases, PMA must deal with not just the formats of the data, but also topics related to device and test equipment data collection mechanism. There are 2 main components needed to come out with the solution, namely data parsing and data pipeline.
This time we will be focusing on the topic of data collected from same manufacturing site that has more than one test lines, each with different tests running (namely ‘type_A’ and ‘type_B’ in this scenario). For example, each unit built potentially involve diverse types of functional-specific equipment. Therefore, it is not difficult to imagine that the data generated by each type of test equipment, or software version, may have different formats or even slight variations of data types or values. For instance, the data may come as binary or structured text. The files may come as a single file for each unit of product tested or a grouping of the units tested within a day or so. Also, in this scenario, we are also being challenged to display two separate test types under the same template.
Let us look at one of the scenarios discussed below.
Multi-Test-Lines
In a production line, multiple test lines within the same site (refer to Figure 1.1) are quite common. To approach this scenario, data must be prepared from the source – tester. For instance, a header info named parser_type is added to the log headers. Using parser_type tagging, we can tell our parser to differentiate between two or more test lines in this scenario.
Figure 1.1 Multiple Test Lines Scenario (without equipment)
Tester Equipment in Multi-Test-Lines
In this scenario, we are adding in the tester equipment under each test line where we assume that our user is setting up a single tester equipment under one or more test lines[2]. Therefore, parser_type and equipment_id form a unique pair of combinations. Refer to Figure 2.2 below:
Logs format and parser logic
On the tester logs, below figure shows some of the locations where the additional parameters mentioned above that needed to be modified. In our example, the parser will look for ‘alpha_a’ as the test line, as well as equipment that is listed along, in this instance ‘equipment_a01’. Refer to Figure 1.3 below:
Figure 1.3 Additional info ‘parser_type’ and ‘equipment_id’ added to log header
On the parser side, there are also adjustments required since it is the receiving side. The parser drafted the way where it constantly scans for recognizable ‘parser_type’. In this case we will be looking for ‘alpha_a.’ Phrases that are not recognized will be dropped. The logic and pseudocode example of the parser would be as the Figure 1.4 below:
Figure 1.4 Additional info ‘parser_type’ and ‘equipment_id’ added to log header
Data pipeline – from logs to visualization
Now that the data pipeline has been arranged properly, logs can start flowing from testers to Data Collection Agent (DCA) and then subsequently into PMA Dashboard through Extract, Transform and Load (ETL) process (Refer to Figure 1.5 and Figure 1.6).
Figure 1.5 Data flowing from tester (logs) -> parser (logic) -> dashboard (visualization)
As a summary, parser plays a crucial role in the upstream section of our data pipeline. It is interfacing with data from various sources and transforms them into useful outputs for the next steps in the process (refer to Figure 1.6). The parser needs to know how to ignore and handle the portion that do not have a pattern and still able to process them appropriately. This requires a level of experience and expertise from the team creating the tools necessary to achieve the desired outputs in as cost effective and efficient as possible. There may be scenario that duplicated data might occur in the data causing the failure of the source data processing. For example, some data may be correct in terms of format that fits within the specified criteria, but in actual, it is duplicated data from the source. Instead of just sending it one time, the source is sending it multiple or even thousands of times a day. This is not something that the parsing of data alone may be able to catch without some human intervention.
There will be some level of customization required for each data parsing scenario. It is, therefore, reasonable to expect that each implementation will have some reusable components while others will require certain parts of the data parsing process to be recreated from the ground up. The effort and turn-around time can impact the cost and time for each implementation.
PathWave Manufacturing Analytics (PMA) is a backed by an experienced team of people at Keysight, who can help you achieve the goal of bringing in data from your manufacturing test equipment and software, for analysis and presentation, to help you to gain the necessary actionable insights that can be used from an operator on the line to the highest senior management levels of the business organization. PMA includes a scalable architecture that can grow with your business. It comes with regular software updates that will allow you to manage the changing business or production requirements in your environment. PMA will free up engineering and development teams to allow them to work on the software and tools that are valuable to your manufacturing lines and business, as well as save time on the production floor with its actionable insights.