Unoffical empeg BBS

Quick Links: Empeg FAQ | RioCar.Org | Hijack | BigDisk Builder | jEmplode | emphatic
Repairs: Repairs

Topic Options
#296020 - 29/03/2007 19:47 using GPS to record a trip
kilbasar
new poster

Registered: 29/03/2007
Posts: 9
So, here's my plan. I'm doing a cross-USA road trip over the summer with some friends. Being the geeks that we are, we're bringing along a GPS, and plan on recording our trip so we can upload it to google earth, geosynch our pictures, fun stuff like that. I have a GPS unit (Garmin eTrex Venture) that does your standard NMEA output to the serial port. I want to record the data it outputs so that we can later upload it to a computer and play with it. Using a laptop, this is easy, as there are all sorts of GPS apps that let you configure every minute detail of what you want to do. I know I won't be able to do all that on the Empeg, but I'm pretty sure I can accomplish everything I want just by recording the raw data that comes in from the serial port. I can later parse it on the computer and do whatever with it. Seems simple enough, but i'm not sure how to go about it. Would running a shell script that just did something like "cat /dev/ttys0 >> gps.log" work? My other problem is that according to my (probably wrong) calculations, assuming 115 kilobits/sec coming in thru the com port and 12 hours of driving a day (an overestimate), I could end up with as much as 50 gigs of data if it's always recording (the trip is 6 weeks long), which is too much for me to set aside. Is there any way to only record data every 10 seconds, with the data still being useful? I don't know much about NMEA, so I'm not really sure how that works. And although I can hack my way through shell scripting, if it turns out I have to write REAL code, I'm pretty much screwed. I appreciate any input. Thanks!

Top
#296021 - 29/03/2007 19:58 Re: using GPS to record a trip [Re: kilbasar]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
NMEA is 4800bps, so that would be less than 5% of your original estimate based on 115kbps, so maybe 2GB.

That said, nothing is mounted r/w on the empeg while it's in the car. You could make it mount something r/w, but you risk corrupting data when you shut it down.

Also, the empeg will spit out data on the serial port on boot. It probably won't brick your GPS, but I don't know that I'd trust that. I think Mark has added some stuff to Hijack to allow app support for the serial port, which is usually the console, but I don't know how that all works.

Honestly, I'd be inclined to either use a laptop, if laziness is the top concern, or get some single-board PC with a big-ish flash disk running Linux if it's not.
_________________________
Bitt Faulk

Top
#296022 - 29/03/2007 20:00 Re: using GPS to record a trip [Re: kilbasar]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14486
Loc: Canada
Most (all?) consumer grade GPS units report position over serial once per second. So the amount of data to be recorded is actually quite small, even over a long period.

Top
#296023 - 29/03/2007 20:40 Re: using GPS to record a trip [Re: wfaulk]
kilbasar
new poster

Registered: 29/03/2007
Posts: 9
Hmm... so recording everything isn't that much data. That's good, and simplifies things. But yeah, I forgot that the empeg keeps the hard drives read-only, and spun down most of the time. I guess there's probably a way to hold the data in memory until a certain amount has been recorded, then write it in chunks to the hard drive. That would help with keeping the drive from spinning all the time, which wouldn't be good for it. How much memory is there for apps to use? Probably not much. As for the R/W thing, that's a bit trickier...

Top
#296024 - 29/03/2007 20:43 Re: using GPS to record a trip [Re: kilbasar]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31583
Loc: Seattle, WA
I notice that was your first post. Welcome! Did you just buy the empeg?

If your upcoming trip is Very Soon, I'd say forget it and do it all with a laptop. Getting the kinks of the system ironed out for an empeg-based application will be tricky and take some time and some dry runs, with questions answered by the BBS members here.

Your biggest issue is that you'd need to do some hacking to enable a partition on the player to stay read-write the entire time. I think there's an extra partition that might be big enough that you could format and use for your own nefarious purposes. But it'd be tricky to configure.
_________________________
Tony Fabris

Top
#296025 - 29/03/2007 20:53 Re: using GPS to record a trip [Re: tfabris]
kilbasar
new poster

Registered: 29/03/2007
Posts: 9
Actually, I've posted here before (only once or twice), but I guess I forgot my login, so I'm brand-new again. I've had my empeg for a little over 2 years now, and have loved every minute of it. Although I'm not sure if I like it more for its actual features, or just for how much it impresses people who get in my car. My trip isn't until July, but I'm trying to get things ready well in advance so that it'll all work out. Originally I planned on just using a laptop, but then I thought "Y'know, I've already got a linux computer with a serial port in my car, why don't I just use that?" It also seems like a fun project.

I couldn't just re-partition one of the drives however I like? I don't mind losing my data, as I can put it all back fairly easily.

Top
#296026 - 29/03/2007 20:55 Re: using GPS to record a trip [Re: kilbasar]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14486
Loc: Canada
Quote:
Hmm... so recording everything isn't that much data. That's good, and simplifies things. But yeah, I forgot that the empeg keeps the hard drives read-only, and spun down most of the time. I guess there's probably a way to hold the data in memory until a certain amount has been recorded, then write it in chunks to the hard drive. That would help with keeping the drive from spinning all the time, which wouldn't be good for it. How much memory is there for apps to use? Probably not much. As for the R/W thing, that's a bit trickier...


The whole job can be done with a moderate bash shell script running on the empeg. You can use the Hijack ;@EXEC_ONCE feature to (re)launch it at boot. The script could create a ramdisk device, and then use dd to copy from the serial port to a temp file on the ramdisk.

Each time the file hits a defined size (eg. 1MB, determined by parameters to dd), the command would exit, and the script would then continue: remount the disk RW, use cat to append the temp file's contents to a master log file, and then set the disk back to RO, nuke the temp file, and loop again.

The tricky bit is ensuring the drive gets set RO before any power-off of the player, as otherwise a possibly lengthy fsck will cripple it at next boot. There's a config.ini setting somewhere that controls power-off after the ignition turns off, and one could tie into that.

This could take a couple of hours for a reasonable UNIX script writer to get working correctly.

Me? I just use a laptop for stuff like this!

Top
#296027 - 29/03/2007 21:03 Re: using GPS to record a trip [Re: mlord]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
Actually, something like this would be a great thing to use ext3 for.
_________________________
Bitt Faulk

Top
#296028 - 29/03/2007 21:13 Re: using GPS to record a trip [Re: kilbasar]
tfabris
carpal tunnel

Registered: 20/12/1999
Posts: 31583
Loc: Seattle, WA
Quote:
I couldn't just re-partition one of the drives however I like? I don't mind losing my data, as I can put it all back fairly easily.

The player is picky about its partitions when you apply firmware upgrades. If you mess with the partitions, you might mess that all up. Supposedly there is a spare 32 meg partition reserved for stuff just like this. I don't know what the details are regarding how to access it.
_________________________
Tony Fabris

Top
#296029 - 29/03/2007 21:36 Re: using GPS to record a trip [Re: tfabris]
oliver
addict

Registered: 02/04/2002
Posts: 691
Why not just pickup one of those nifty IDE>CF adapters, grab a decent size cf card, and setup the scripts to write directly to the flash device?

Then after the trip they could use the card for those nifty new hijack features that are in the works
_________________________
Oliver mk1 30gb: 129 | mk2a 30gb: 040104126

Top
#296030 - 29/03/2007 21:52 Re: using GPS to record a trip [Re: oliver]
kilbasar
new poster

Registered: 29/03/2007
Posts: 9
Quote:
Why not just pickup one of those nifty IDE>CF adapters, grab a decent size cf card, and setup the scripts to write directly to the flash device?


that could work, except i'd have to take out one of my hard drives (i have a 2-drive setup)

Quote:
The tricky bit is ensuring the drive gets set RO before any power-off of the player, as otherwise a possibly lengthy fsck will cripple it at next boot.


The GPS will mostly be running during long, well-defined drives, so I wouldn't mind if I had to manually start/stop the script (as long as it's something I can do from the player menu, ie something in the Hijack menu). This way, the script can set it back to RO before it closes, and then I just have to make sure I close the script before I power off the player.

Quote:
Supposedly there is a spare 32 meg partition reserved for stuff just like this.


According my revised calculations (this time somewhat based in reality), the NMEA data will come to about 2 megabytes per hour. Not a lot, but when you're talking about 6 weeks of driving, 32 megs won't be enough. On the other hand, assuming I go the path of making a RAM drive for temp storage, even if the RAM drive is only 1 meg in size, that means I'll only need to write to the hard drive every half hour or so (plus right before the script closes). In which case, it doesn't seem like it would be too risky to have the normal data partition set to RW for a couple of seconds every half hour. Does that make sense?


Edited by kilbasar (29/03/2007 22:10)

Top
#296031 - 29/03/2007 21:57 Re: using GPS to record a trip [Re: kilbasar]
Attack
addict

Registered: 01/03/2002
Posts: 598
Loc: Florida
I've seen devices that you just plug in a USB drive into an it writes the logs to the drive but was unable to find that in my 15 seconds of searching. I did find these:
http://www.linkspoint.com/sdl.asp

http://www.thinkgeek.com/gadgets/security/8f90/
_________________________
Chad

Top
#296032 - 30/03/2007 06:58 Re: using GPS to record a trip [Re: kilbasar]
julf
veteran

Registered: 01/10/2001
Posts: 1307
Loc: Amsterdam, The Netherlands
Are you stuck with the GPS you have? I am doing the logging using my TomTom 700 - remember it runs Linux too.

Top
#296033 - 30/03/2007 11:55 Re: using GPS to record a trip [Re: kilbasar]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14486
Loc: Canada
Quote:

According my revised calculations (this time somewhat based in reality), the NMEA data will come to about 2 megabytes per hour. Not a lot, but when you're talking about 6 weeks of driving, 32 megs won't be enough. On the other hand, assuming I go the path of making a RAM drive for temp storage, even if the RAM drive is only 1 meg in size, that means I'll only need to write to the hard drive every half hour or so (plus right before the script closes). In which case, it doesn't seem like it would be too risky to have the normal data partition set to RW for a couple of seconds every half hour. Does that make sense?


Yes, that makes sense.

As Bitt suggested, you can also simplify handling of the RO/RW status by installing one of mcomb's ext3 enabled Hijack builds (or build it yourself from the standard Hijack source code -- it's just a config option that needs to be enabled). With ext3, the drive could be left mounted RW without much chance of harm, giving less risk of something bad happening should the empeg power down suddenly.

Cheers

Top
#296034 - 30/03/2007 12:08 Re: using GPS to record a trip [Re: kilbasar]
peter
carpal tunnel

Registered: 13/07/2000
Posts: 4174
Loc: Cambridge, England
Quote:
According my revised calculations (this time somewhat based in reality), the NMEA data will come to about 2 megabytes per hour. Not a lot, but when you're talking about 6 weeks of driving, 32 megs won't be enough. On the other hand, assuming I go the path of making a RAM drive for temp storage, even if the RAM drive is only 1 meg in size, that means I'll only need to write to the hard drive every half hour or so (plus right before the script closes). In which case, it doesn't seem like it would be too risky to have the normal data partition set to RW for a couple of seconds every half hour. Does that make sense?

If you're prepared to do a bit of coding for it, you can eliminate remounting altogether by pre-preparing a file of large-enough size, working out which disk sectors the file occupies using ioctl(FIBMAP), and writing to those sectors directly in the partition block device. This reduces the risk of a power failure hosing your filesystem, though of course it increases the risk of a bug in your own code hosing the filesystem...

Peter

Top
#296035 - 30/03/2007 12:19 Re: using GPS to record a trip [Re: peter]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
From OP:

Quote:
And although I can hack my way through shell scripting, if it turns out I have to write REAL code, I'm pretty much screwed.
_________________________
Bitt Faulk

Top
#296036 - 30/03/2007 14:16 Re: using GPS to record a trip [Re: kilbasar]
siberia37
old hand

Registered: 09/01/2002
Posts: 702
Loc: Tacoma,WA
If cost isn't much of a concern just get one of these for $150 and your set.

Top
#296037 - 30/03/2007 15:38 Re: using GPS to record a trip [Re: siberia37]
wfaulk
carpal tunnel

Registered: 25/12/2000
Posts: 16706
Loc: Raleigh, NC US
Or, since he already apparently has a serial-out GPS receiver, one of these for $60 and an SD card.
_________________________
Bitt Faulk

Top
#296038 - 01/04/2007 21:56 Re: using GPS to record a trip [Re: siberia37]
kilbasar
new poster

Registered: 29/03/2007
Posts: 9
Quote:
If cost isn't much of a concern just get one of these for $150 and your set.


cost is a concern. i already have the GPS, i already have the Empeg, so why spend another $60 for a device I'll only use once?

From what I've heard here, it seems like I should be able to get this working with a not-too-complicated script. I'll be out of town for the next two weeks, but will start writing the script when I get back. Thank you all for your help, and I'll be sure to bug you again once I start the code writing!

Top
#296039 - 02/04/2007 00:13 Re: using GPS to record a trip [Re: kilbasar]
jaharkes
enthusiast

Registered: 20/08/2002
Posts: 340
Loc: Pittsburgh, PA
The etrex venture already logs a track, so really the only thing you need is something like gpstrans on the empeg to download the log once a day or so.

Oh, and bring lots of batteries.


Edited by jaharkes (02/04/2007 00:25)
_________________________
40GB - serial #40104051 gpsapp

Top
#296040 - 02/04/2007 00:33 Re: using GPS to record a trip [Re: kilbasar]
mlord
carpal tunnel

Registered: 29/08/2000
Posts: 14486
Loc: Canada
Quote:
i already have the Empeg, so why spend another $60 for a device I'll only use once?

Exactly. And where's the fun in just spending one's way to a solution?
Quote:

From what I've heard here, it seems like I should be able to get this working with a not-too-complicated script. I'll be out of town for the next two weeks, but will start writing the script when I get back. Thank you all for your help, and I'll be sure to bug you again once I start the code writing!


I think there'll be a few tricks and traps along the way with this, but just keep updating us here as things progress, and I (and others?) will help you out with it.

Cheers

Top