And my experience has been in the private sector, where no government secrecy levels are involved and no governmental regulation of the sort they are dealing with. Also I am working with companies whose best interest is served by getting things up and running as soon as possible.

So, am I shocked and surprised... Yep, by the fact that they got it up and going at all. Not by how long it took them.

You've got a couple approaches to use when dealing with the myriad tangle of data the government undoubtedly has spread among its organizations.

Extract-Transform-Load (ETL) builds giant databases from the individual databases used by the departments, but it's not real-time--as the name implies, somewhere along the line, a separate process has to extract the information from the disparate data sources, transform it to match the master database, and then load the master database. Typically, it's on a nightly or weekly basis. Still, to build an ETL program and its "data warehouse", you've got to do some months of analysiz, design, coding, and testing--and even then, it's not real-time. And after you have the data warehouse (the big database you have at the end which contains all the information from all the sources), you have to build (and test) the applications and queries that will extract the (old) information from within it.

Enterprise Application Integration (EAI) works from a different approach--it massages the information from disparate sources so that existing applications and queries can use it. For example, the EAI approach would make it easy if you have an application that draws information from an existing source, the EAI middleware would handle communications with the different applications and make it seamless, but the government would have to do create different middleware for each application accessing the data.

Enterprise Information Integration (EII) is a newer technology that sits between any application and any number of data sources and abstracts information so it can take any databases and present them any way the applications want. Although it's probably the best approach, it's the newest and the technologies aren't mature or easy to use.

So your idea, James, is conceptually easy, but difficult to implement. Within a couple of years, though, it will be automatic.

---