Quantcast
Channel: LabVIEW topics
Viewing all articles
Browse latest Browse all 67075

Fast processing of mixed representation binary files

$
0
0

Hi Everyone,

 

I see multiple ways of tackling this but I'm looking for the fastest approach as my data set is very large....

 

The issue:

I have a binary data file holding 2D data.

It encodes 200+ differnet "columns" that are repeated in time (sampled)

The data contains mixed data representaitons: a mxiture of  U8, I8, U16, I16 etc.

They are all regullarly repeated in a known file structure (660 bytes per "line")

I'd like to gnereate the 200+ differnet 1-D arrays form the file each using the correct data representaiton (or a sub-set of the columns).

 

I can load the file in using binary file read and I specify U8 as the data type . I can then rediension to the correct 2D array.

 

I'm now stuck on the fastest method to process the columns of data (1-2 bytes wide) into the corect numeric representaiton 1-D arrays (2x U8 to i16 etc.).

Scanning byte by byte would be very slow.

 

any suggesitons?


Viewing all articles
Browse latest Browse all 67075

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>