The software I use at work is SQL, provided by our Corporate Gods. It is badly conceived, poorly written, incompetently executed, but I have no choice but to use it. Every "upgrade" they force on us makes things worse. I have more experience in my area of work than any other employee in the corporation which has, I'd guess, about 50,000 employees, so I know whereof I speak. I would estimate that my workload has increased by 40-50% in the two and a half years we have used this software, producing the same output but at a considerable lesser quality. Because of this software I have actually changed my retirement plans, I am going to retire three years sooner than originally planned.

One of the quaint aspects of this software is that there is no way to purge obsolete data. I mean, no way whatsoever. Any bit of data entered in the system, no matter how trivial or time-limited is retained forever. So the database(s) in the system continue to grow without limit. I don't know how many different inter-related databases there are, or maybe, horrors, it's all in one big flat file with lots of indices. Beats me... but I do know that just the data that I put in is probably generating between 2,000 -- 3,000 new bits of retrievable information every day. And I'm not the only one entering data.

So now the question.How big can an SQL database system get before it collapses under its own weight?

I'm seeing hints of impending trouble already. The system is becoming less stable, the last update to the software ended up causing random bits of corruption and data loss here and there. The technical support for this software is superb, the best I've ever had for any product, hardware or software. They tell me not to worry.

What do you guys think?

tanstaafl.

"SQL" isn't really quite enough information to answer your question. "SQL" is a specification for how to interact with a database, not a product. Most databases are accessible via SQL queries.

However, Microsoft produces a product called "SQL Server", which is often abbreviated to "SQL", which is quite annoying.

Assuming that the product you're talking about is Microsoft SQL Server, I would doubt that the amount of data you're talking about is that big. Verizon seems to hold the record at over 5TB and over 33 billion records.

Oracle is another very popular database system, and it has a record of 100TB of data in 385 billion records. And that's not even the largest database in that survey (330TB for size, nearly 3 trillion rows, in different databases).

Of course, even that doesn't take into account the application that you're using that accesses the database. And that is, more likely, where any problems you're seeing lie. And that has virtually nothing to do with the database system itself.

Well, no one who's aware of SQL separate from SQL Server abbreviates it that way, but I've heard a lot of non-technical people refer to it that way. It's interesting how people who don't know anything can confuse those who do.

For example, what would you assume Doug is talking about here?

For example, what would you assume Doug is talking about here?

I can answer that...

I say SQL because I see that acronym popping up in error messages, not because I have the foggiest idea of just what the software really consists of.


I'm basically looking for reassurance that whatever database system we are using is not going to implode under its own weight due to what I had thought was a large and ever growing size. Apparently what I thought was large was really pretty small potatoes. Probably not more than a couple million pieces of data in it. Wal-Mart probably adds that much every day to their database.

tanstaafl.