Data Storage: A Case Study
My first tech job was with a company that created software to put periodicals online. Most of the people involved had been involved in the publication business, so we had domain knowledge and great contacts. Most of the technical work was the “seat of the pants” type scramble, but it was a good scramble because we cared about doing a good job and providing quality customer service.
One of the decisions that had to be made was how to store news and other content from the periodicals. It wasn’t just a storage place; the content had to be uploaded by staff at the periodical, stored on the server, retrieved and added to web coding every time someone wanted to read a page. This was in the 1990s. This model is still how most websites work. The questions of methods still go back to the basics every now and then, “What is the best way to store data for websites.
In this scenario, before the database tools that we have now were available, the programmers decided to use flat text files to hold news articles data. The newspaper clients would upload a text file in a specific format and software would parse it and create web pages on the fly.
Understanding the Structure of the Data
At the start, data we needed to store was rather simple. Articles had 1) a title, 2) a byline (author info), 3) the article text. As far as the folks working for the periodical, that was the structure. For us, it was more complicated, because our software had to add HTML code to allow formatting for the viewers. So, besides a method of storing the data, we needed a way for the software to be able to tell the difference between the title, byline and text for formatting purposes.