The Great Data Dilemma

I love my new job - it is great for me in so many ways. But perhaps the way that it is worst for me, but I am best for it, is the insane mess that is its data storage. I swear I am going to clean this mess up, if it takes me a decade to do it, which it probably will. Just some of the highlights (lowlights?) include:


1. Obsession with hard copies.

For being such a small business (there are only 8 employees with computer access) my work has a very progressive server backup schedule. In addition to complete backups done every night, there are incremental backups done throughout the day. But that isn't enough for the owners to feel safe about our data, and so every document that is on our servers is also in our filing cabinets. My office alone has three filing cabinets (with a forth coming soon) plus about 25 three inch binders that are all just old document storage; every office is like this because we have 20 years worth of everything the company has ever done.




2. Absurd file structure

I know a lot of companies are guilty of this, but my current one is one of the worst, that is having files on the server laid out in a way that is completely nonsensical. All of the transgressions that you can have in file structure are committed here, from poorly named folders to empty folders to folders that contain a single sub-folder to data that belongs in one place being spread out over several places. I have already done some work to cleaning this up, but I don't see myself attacking any of the biggest issues for at least a year or two.




3. Drawing naming system

While a lot of files have very poor names, we have so many drawings and are so dependent on them the absurdity that is our system for naming drawing stands out as the worst. The structure was borrowed from a company where the two owners used to work together, at a time that drawings were done by hand. I can see how it could work in a filing cabinet, but it is a very poor structure to use with electronic files (because it makes it very difficult to find the file you want). I have already come up with a good alternative, but I am not going to proposing it for some time, both because I know they are quite enamored with the current structure and because I know the work involved in changing over would be gargantuan.

4.  Interlinking files

One of the reasons that changing the drawing naming structure would be insane is that the company loves having Solidworks files refer to other  Soildworks files. Sometimes interlinking the files this way is a perfectly sensibly way to ensure that any future changes propagate through all connected files, but it is used so aggressively in this company that it can have the opposite effect, that changes that are meant to affect only a small number of parts could make unintended changes elsewhere.




5. Poor choice of file type

My company has a whole lot of data stored in Excel files and Word files. This data is so full of relationships that even the Excel files are poor choices; what is really needed is a database. I decided early on that this was going to be the first of the data management problems that I tackle, but it is such a huge problem that knowing where to start has been troublesome. Originally I was going to start with moving the part numbers spreadsheet into a database both because it would be easy to do and because so much of the other data really should be linked together via the part numbers. However, I have since decided that the best starting point is with smaller data sets that are primarily used by myself; I suspect that there will be no resistance when I suggest moving those over, and once I have them up and running, they can be used to demonstrate the advantages of a database in selling my bosses on changing other information into a database format.