rss

Artictles

Hi, thanks for visiting this page. Here I write about technology, in-depth analysis of some problems, sport management, among others. If you like these things, you will be glad to read the articles. Also, you can comment or ask questions whenever you want. I will try to respond as soon as possible...

Import a 900MB log file in MATLAB allocate around 32GB in RAM Bookmark

If you have enough RAM memory in your system, you could take a look to this process.

The application Windows Task Manager show the requisition memory before parsing the data.

image

After the memory was allocated, the parsing process start (basically, read every line and parse three number text ASCII to internal number representation).

image

Next time will be necessary to record a binary log!

Update

After waiting one hour during parsing process, only 9% was completed. Once the process is cancel, the memory is free.

image 

File properties

  • Type: text (.txt)
  • Lines: 46.771.712
  • Size: 934.604.800 bytes
  • Fragment:

249666;987154022;0
250334;987404356;0
250000;987654356;0
250000;987904356;0
250000;988154356;0
250000;988404356;0…

blog comments powered by Disqus