Well one thing to consider is that nanosleep calls are at the mercy of the scheduling policy, and the player uses realtime scheduling. Unless you switch your app to the same scheduling policy (SCHED_RR I believe) you aren't guaranteed to get any CPU at all.

Here's how I do it in emptriv, which is a little slow with the default scheduler:

#include <sched.h> 



#define PRIORITY 30



struct sched_param  sched_param;

int iPid, rc;



iPid = getpid();

rc  = sched_getparam(iPid, &sched_param);

sched_param.sched_priority = PRIORITY;

rc = sched_setscheduler(iPid, SCHED_RR, &sched_param);	

rc = sched_setparam(iPid, &sched_param);

.

I suspect that if you switch to SCHED_RR, you'll get more consistent timing, though you might have to tweak the priority. Anything beyond 30 and you might start stealing too much from the player. At least those were my results.

Not sure this is the problem, but it's one possibility.