We have a single (customer) point of access at this point, but have not considered "choking" the rate at that point as updates are taking place. I'll have to look into this.
What do you mean by buffered processes?
I think you broke it down very well to say there is two distinct problems here. The most impactful of the two is the slow source problem. The feed out of there requires multiple requests to satisfy, and each request can take on the order of 5 seconds each, which can be problematic with hundreds of requests needed to compose a response. This wouldn't ordinarily be an issue because we can throw multithreaded requests at it, except the source tends to slow down even more when that's done. So for this reason, we're looking at a reverse proxy of some type. I'm thinking of squid operating in httpd-accelerate mode perhaps.
The type of responses we ship out are complete "answers" that aren't easy to break down in terms of the speed of the individual sources. So if a response requires Sources A, B, C, D, E ; and the response times are 1, 2, 3, 4, 5 respectively. The response has to be complete and whole, and will end up taking up as long as the slowest response. Distributing the sources (as opposed to collating them into a single large db) doesn't make a lot of sense because the ultimate origin of the sources are from outside our systems. But I can see what you're getting at, if the sources are partitioned, then the updates can be uninterrupted and access to data in the process of being updated is controlled by each source independently. Correct me here if I'm misinterpreting or not reading through correctly.
Calvin