HDD full - what to do next?

mhedges

I've finally filled the internal 2 TB HDD on my photo PC and I'm trying to figure out the best way to add storage while still being able to access all my images and be able to backup without too much hassle. Currently I am using external HDD's for backup via the programs that come with the drives. Operating system etc. are on a separate SSD.

My typical workflow is to copy from camera card to the 2 TB HD, then edit and export to the SSD.

From what I can tell my options are:

Add an external RAID type system ($$, and possibly slow?)
Replace the 2TB drive with bigger internal HDD
Add a second internal drive (have to check if there is room etc.)

Any suggestions here?

Thanks!

photobunny

I use a 8 drive Synology RackStation on 10 Gbps ethernet with 10 TB Seagate Iron Wolf drives. Because spinning rust is rather loud, it being on a rack lets me move that noise out of the office and into the cupboard.

Even on 1 Gbps Ethernet you'll be able to edit photos just fine from one of these and you can start of with one drive and just live plug other drives as need be and they'll be added to the pool. A major plus is virus scanning on the server and I have it set to auto backup to AWS S3 Glacier.

When you get a new machine the data doesn't need transferred, you just connect to the network and it is there all ready and waiting for you. If a drive fails you don't lose data and can just replace the dead drive and in the event of catastrophe you plug in your new Synology NAS and tell it where your S3 Glacier is and it'll rebuild back to as If nothing happened (and hopefully your insurance covered the hardware without fuss).

WestEndFoto

The weakness of this approach is that a RAID is not a backup. It is just a complicated way of improving reliability. This complication is less of an issue if you are really technically proficient.

I currently have two Synology 8 bay NAS with 8TB Western Digital Gold Drives in each bay. I have 24 of these Gold Drives – 8 in each NAS and 8 in a safety deposit box. This is bullet proof and quite expensive. But I have a huge volume of data and backing this up online would be thousands per month. At work, our backup is even more robust.

I think that this is overkill MHedges. You only have a single 2TB drive.

First, back to basics. First, you are not backed up if you do not have off premise backup. I am surprised how many people with RAIDS think they are backed up and they get defensive when I tell them that they are not backed up, but merely protected from one or two drives failing. Second, you need some redundancy. If you have two sets of offsite backup, it is less likely that a “patient” attack, perhaps with ransomware, will compromise all of your backup. Third, you should think about keeping your backups totally disconnected from the internet. For example, at work we use Microsoft Azure for offsite backup but I don’t fully trust it as it is “online”. We have additional levels of protection including encrypted backup drives in the company’s safety deposit box which are rotated once per week.

As I said, MHedges, your needs are pretty simple. I would buy four 4-8TB external drives with a docking station depending on your anticipated needs. Western Digital Red drives would be adequate and reasonably priced. One is your “main” drive in a docking station. The other is your backup in another docking station. The other two are in your safety deposit box or some other secure OFFSITE location – giving you three backups that you rotate on a regular schedule. If you want to get cute, buy Syncback Pro and run it every night when you go to bed.

Ton14

My computer has 3 SSD disks, 1 x 1tb M.2 for the OS and Software, 1 4tb m.2 for photos and 1 Samsung 870 QVO of 4 tb for the rest. All disks are about half full and 2 years old by now. They are continuously synchronized with "Synology drive" to the NAS.

Daily backup of the NAS to an external USB drive with Synology's "Hyperbackup".
Also the NAS is daily backed up to the Synology C2 cloud, @WestEndFoto , this is my real external backup of the NAS and all my files. you are right RAID is not a backup.

The big advantage of this is that it happens at night, is fully encrypted and automated. The "Hyperbackup" of Synology I have set so, that I can even go back 6 months.

Furthermore I use my old 4tb disks for a monthly rotating backup. Recently I bought 2 LaCie 2.5 inch 5tb disks, which I now use for that. Just run the backup in the evening once with the "SyncbackPro" software. I have also set it with "versions", which automatically creates a hidden folder with the old files, so I can also go back 4 versions.

As workdrive for photos I have the very small and fast Sandisk 2tb SSD, here are all my current photos and Capture One sessions, so I can work with it on any computer.
When a session is finished I import it into the big Capture One catalogue on my desktop.

Setting everything up once took quite some preparation, but now I don't have to worry about it anymore, Hyperbackup from Synology is the best backup program for me at the moment, it also has a "Hyperbackup Explorer" for the desktop, so the files on the external USB drive and in the C2 Synology cloud are directly accessible.

Im my PC I have a PCI slot free, where I can put in a card for an extra M.2 SSD, maybe you can too.

I wish I had 10 Gbps ethernet as @photobunny

photobunny

@Ton14 10 Gbps is great but expensive. Though some of my home backbone have 25 Gbps links too.. I have yet to try these due to cost.

2.5 Gbps is however a great half step letting you get 250 MB/s transfers from a NAS which is more than sufficient to work with on photos and much cheeper with a lot of consumer grade equipment and even Cat 5e cables still working without a fuss.

Also to add. RAID 5/6 is going to give you a good balance of performance and fall over, but aye, as I said you should connect to some backup. AWS S3 Glacier is exceedingly cheap to store hundreds of TB's of backups. It only charges you any reasonable amount when you are recovering the files.

Ton14

@photobunny Yes I looked into the 2.5Gbs what seems enough for me as a home user to work with. In the coming months I will get fibre in my neighbourhood and then I will install 2.5Gbps, if you know more about 2.5Gbps please give some info.

I checked AWS S3 glacier for my situation and I don't know what to choose, I need a cloud storage of maximum 3tb, at the moment 1tb. I have a Synology DS218+ with 12tb, enough for many years for me.

On the pricing page I found:
- S3 Standard - General purpose storage for any type of data, typically used for frequently accessed data, $0,0245 per gb, $73,50 per month, $882, too expensive for me.

- S3 Glacier Instant Retrieval*** - For long-lived archive data accessed once a quarter with instant retrieval in milliseconds, $0,005 per gb and that is $18.- per year, so very cheap

- S3 Glacier Deep Archive*** - For long-term data archiving that is accessed once or twice in a year and can be restored within 12 hours is $ 0,0018 per gb, that is $64,80 per year, also very cheap.

The only thing I compare it with is the C2 cloud storage from Synology, 3tb is €210.- per year. I can access this from my desktop on file level, when I want.

At the moment I have a 1tb subscription for € 72.- per year, because there is no need for me to put all my data in a cloud. I can change this subscription any time, but a cheap long-lived archive is of course very welcome.

Synology Hyperbackup has also a possibility for a direct connection with AWS S3.

mhedges

Thanks all. I do have offsite storage via external HDD's in a safe deposit box. But of course that requires some effort on my part to update.

Still not sure which way to go. I agree that RAID etc. seems overkill for what I need. I'll look into the docks.

Ton14

@mhedges I totally agree with you that a NAS is an overkill for just photos, if you are a regular home user like me. Backups like I have with Syncbackpro set up require hardly any effort. Just click the external USB disk to the computer and click on the backup, daily time is 2-10 minutes per disk and often it does not need to be daily. Also once a month I use old 4tb disks, which I have from previous computers anyway. This way has always worked well for my home use.

My NAS however has many additional benefits, it is my own cloud, because Synology has made it very easy for users like me with many applications, available for desktop Mac, Windows, IOS and Android, everything fits seamlessly well secured together and can be accessed from anywhere.

Synology drive is my Cloud, Synology Photo all photos, Secure Signin is 2FA, with Synology Video and Synology Audio I have all my videos and music on mobile, tablet and desktop.

With Synology Hyperbackup I can restore my accidentally deleted files anywhere. I also see now that there is a dedicated "Glacier Backup" package for business use.

If you want you can create your own website hosting, Synology Office, Mail server, Note Station, just too much to mention. Slowly, but surely, I started in 2012 using more and more now, which ended up being much cheaper for me and the most important thing is that I am completely independent and have everything under control.