So, it's trivially easy to get a particular sensor's data, or all data for a particular hour/day, but getting all flows from a particular IP over a wide time range is a brute force chug through the files.
It'll still be a huge chug no matter how much SQL middleware you put between you and it. IMO the only way to get a sane outcome here is to write scripts to import the data into a real database. Fortunately this duplication isn't a problem, as the data never changes once it's generated (right?).
Peter
Previous Topic
Index
