I will talk about different data storage methods and their advantages. I will not name companies. A search of product and service types will give you a wide range of options. When working on a project, some of the files will be kept on your local machine. To be proficient after some operation(s) are performed, you can move files to a remote location to stop your computer from slowing down.
The Desktop
- Have a large (2TB+) solid state drive (SSD) for Windows and other programs.
- Have a second solid state drive for file storage. We have added multiple SSD’s for expanded storage. Prices are low and performance is required for data modeling and photogrammetry.
- Keep things on the drive as the project goes along. Remote storage is good, but a lot of files may need to be brought back to the SSD’s to do some more work.
I have not mentioned backup of your local machine yet. We will cover local (somewhere you can get to easily) and cloud backups. Remote backup is also another option.
Local Backup
The easiest way to make sure things are safe is to back up your hard drive and put it in a fire safe. A process that is time consuming and prone to forgetfulness. I do not recommend this as there are other solutions.
Network Attached Storage (NAS)
NAS is the easiest way to grab something, work on it, then put it away safely. As the name implies, you have a large amount of storage that is connected to your network. It is a drive that is labeled as a letter, I use “N” for simplicity. The drive itself acts as a small computer with a disk array for redundancy in case of partial failure. Transfer speeds are generally good making it easy to retrieve and save data. There are other benefits as well.
Another NAS drive can be placed offsite, giving you added protection in the event of complete destruction of one of the locations or its drive. I did this for our firm years ago and had a host of issues getting things set up. There are a number of independent consultants who will do this in a matter of minutes and it is well worth the money. Here is a road map.
- Find a technician to set up a NAS. The brand you buy will depend on what the technician recommends.
- Decide on the amount of storage you want. The technician can help with that.
- You will get two NAS drives. One will go in the office, the other in a remote location that has a fast internet connection.
- Both drives will start out in the office, the initial large data transfer will be easier that way.
- The drives are synced and tested while in one location, this makes troubleshooting easier.
- The second drive goes to the remote location.
- Drives can be set to back up continuously or at a specified time, usually early morning hours when no bandwidth is being consumed
This peace of mind is critical in securing your data information. Also make sure that important accounting data and correspondence gets saved from other staffers and departments in the office. The cost for this is less than you may think, and you can avoid sad stories of lost data and ransomware.
Cloud Backup
Whenever I mention the possibility of using cloud backup, I often get a response related to security. First, platforms like Dropbox are secure and second, the plans and files you are saving are usually public domain if somebody wants to look them up. How tragic is it if somebody sees the CAD for a subdivision anyway? The exception I will agree is sensitive correspondence, financial information, and most internal communication. Use the NAS for that and trust the rest online. Here is the process.
- Decide on the platform and storage limits. Prices can vary widely so shop around. Security is similar with most so do not let that be an issue.
- Decide on what to sync. I use the sync folder as local access as well because the data resides locally and is mirrored to the cloud.
- Depending on your connection to the internet, you can choose to continually update or pick a specified time. This is usually in the early morning hours when you are hopefully not working, and speeds are good.
- A dashboard for the program will keep you posted with the status of a sync.
This easy solution may be the best option due to the protracted setup of a NAS. If you have followed along to this point, there is still the possibility of accounting and correspondence not being saved. Do not delay the install of a NAS too long.
Purging Stored Data
We never really get rid of information completely. After a time, we save a distillation of information and lock it away. You may also want to check with your attorney as to how long they feel you should have full saves as opposed to the reduced density mentioned. Here are some examples.
- Regarding machine control data, we save the latest file we made in the native software and the last files sent to the field. The pdf’s can be saved as well. Sometimes if they are huge, we will strip out pages outside our scope. Remember they are generally available at the agency responsible for approvals. Let them warehouse it. We do not need the original CAD files.
- Photogrammetry jobs get saved as the point cloud and GEO-tiff. Raw images and initial surface production can go away. With the point cloud we can generate what is necessary in case we need to go back. That will save a ton of space.
- Takeoffs can usually be the program file we used for the numbers. No reports need to be saved as you can run them again. We save the pdf sheets associated with the takeoff as well.
- Job notes and correspondence can get saved in total. They are almost always copies of emails and letters that take up little room. I have spent more time than it is worth trying to decide what should stay and the space saved is minimal.
Well, there you have it. A comprehensive plan for saving and securing data. The pandemic has changed everything and many of you are working at home and need access like as if you were in your office. When I started this company, we all worked remote and were on the cutting (painful) edge of a lot of the previously mentioned technologies. Things have gotten much simpler and integrating some or all these ideas will give you a piece of mind. There are a lot of horror stories out there regarding lost or stolen data. Hopefully, you will not be one of them.