Good afternoon.
I have implemented a set of software on a cRIO (NI-9074) that controls units under test, acquires analog data from 85 channels, analyzes the data for failure conditions, and creates a data file each time that a failure event occurs.
The control portion of the system turns lighting ballasts on and off and signals the ballasts to switch between normal (full) power and dim. The dim control for some products uses relay outputs. For other products the dim control is performed using RS-232 communication.
The data acquisition portion of the system samples the analog channels at 10 Hz and "inserts" the data into a ring buffer of 20,000 elements. This portion also determines when the electrical current is outside of the specified range. When the electrical current goes outside of the specified range, an event is triggered to extract the data related to the unit that failed from the ring buffer and write it to a binary file.
The software causes the cRIO to crash after several hours (typically 4 to 8 hours). The OS crashes, not just the software that I developed. I know this because I cannot connect to the system using NI-MAX, Distributed System Manager, or a web browser. I am working to isolate the section of code that is causing the problem. I would like to think that I write perfect code :-). I have determined that I am closing any resources that I open. I am logging any errors to a file. The software is not reporting any errors. I have added functions to report the system resources a few times per second when they are over-utilized. I am reporting CPU load and free physical memory. The CPU load goes to 100% for around 4 seconds each time a failure event data file is written to the compact flash. Is this a potential problem? If so, what is the recommended approach for writing large blocks of data to a file in a way that reduces CPU load?
I am interested to know what would happen if, for example, I try to access an element of an array that has not been defined. I have built and run a test VI to see what happens when I try to retrieve an element at an index beyond the size of an array and to replace a subset at a location beyond the size of an array. It doesn't seem to break anything. Does anyone know of catastrophic behavior resulting from providing an invalid array index?
I am executing the software in ways that exercise only portions of the system to isolate the problem.
Does anyone have suggestions for ways to debug such a system?
Thanks for any suggestions,
Hamilton Woods