Hey folks,
I know it has been quiet around here lately but that's not due to a lack of projects or things to do. I wanted to give an update on some of this work and the new site section.
My amazing partner that has been building things out is nearly ready for our initial release. While he finishes laying down the foundation and erecting the walls, I come in behind him and check to make sure things are all nailed down (QA). We've had a very good collaboration on how things could work, and the truth is most of the heavy lifting is behind the scenes and out of public view - the Content Management System. The CMS allows me to add new content to this area of the site. It's very complex and quite sophisticated.
You've not seen anything pop up here yet because archiving these stacks and stacks of disks actually is quite time consuming. Last weekend I spent a day learning the process from end to end. By the end of the day I'd completed 4 issues of a disk magazine. The following is an example of doing one single issue from Amiga Animation disk magazine, which contains 3 disks per issue:
- Pop each disk in the Amiga and use David Dunklee's "Disk2ADF" program to create ADFs of each disk. You all know how slow that process is. Do that 3 times.
- Scan the front and back of the disk magazine's "covers". The front is what you'd see at the store or when you received it in the mail. The back contains very detailed descriptions of most of the software and files found on each disk. I scan the front and back at 100dpi using a flatbed scanner, which is purely to produce decent but non-printable web JPEG graphics. I then re-scan the back cover at 300 dpi so I can use OCR software on all of the text.
- Import the 300dpi PDF into Adobe Acrobat (online version, which sadly I discovered after multiple tests that it has better OCR results than the stand-alone desktop software!). Convert the text found in the image, then copy/paste all of that text into a barebones text file.
- Correct all of the line breaks and garbled typos - there are a lot, even in 2024. If you've ever seen a text-converted magazine from Archive.org and seen the pages of broken jank (which they don't fix) you know what I'm talking about here.
- Scan the physical disks as a single 100dpi JPEG.
- Go to AmigaLove.com and upload the front cover as a thumbnail, then upload the ADFs and give them all descriptions. Upload the back cover and disk JPEG. Copy/paste the converted text into the content area and fill in all the other text details (subtitles, year published, etc.)
That right there can easily take 60 minutes for a single issue, if not longer. It's not "work work" but it's non-trivial time spent. I have to know I've got a pretty good time window available before jumping into one of these.
Bonus:
There is a very cool program my partner installed and configured that allows me to "scan" an ADF file's disk contents. This is
very cool, because it can output a Disk Preview of 1 disk on a given page (not all 3, like with some disk magazines). A Disk Preview is essentially a
visual output of every file found on a disk.
Any document file becomes visually exposed so that you, the reader, can consume it without even downloading an ADF. You can read every single document file, you can see images found on the disk (IFFs, etc.) and see the directory structures, all within the content page of a single archive. It's incredibly cool and frankly hard to explain. So I will show you below.
Some disks have so many files this preview can sometimes be overwhelming. So we're currently talking about ways I can clean out and edit the file structures to only leave behind parts you may find interesting. This is more time consuming but produces a better product at the end of the day.
Here are some snapshots of some of the content pages to help you see what's coming.
Landing Page
This is the front door of the Software Archive where you can either search, or more realistically
browse and discover rare Amiga content based on the collection type or timeframe you're interested in.
- Note the new link in the site header.
Collection/Disk Type Page
This is the top level of a given collection, if applicable. In this example, this is the top page of the disk magazine Amiga Animation, which then contains every single volume and issue I'm able to attach to it. The only thing this doesn't have, unfortunately, is a way to "subscribe" to a collection so when I add new issues/disks you could be notified. However, in the short term I'll probably add those updates over here in Site News whenever I add a new batch just so folks have a clue there's more to see.
This page currently has the potential to become quite large and heavy as years of issues are added, so we may have to find ways to optimize that in the future. Time will tell.
Issue/Disk Page
This is an example of an individual issue, Volume 4 Issue 2 from 1990 of
Amiga Animation. This is where the real magic happens.
Hopefully now you can start to see why taking the time to use OCR software to pull all of the text off the magazine covers is so important. Not only can you more easily read all of the contents of a given issue, but it's now also searchable. If I just left it baked into the images it would be borderline useless.
Disk Preview
This is an example of some of what's exposed when a Disk Preview is generated from an ADF file, which would be below the image gallery area on the Disk Pages.
Some of the really long text files can create a bit of a scrolling challenge, but it's a small price to pay for the amount of content this makes available.
This will be especially valuable for single-disk uploads like User Group monthly disks and individual stand-alone shareware/freeware disks. It will save everyone so much time to literally examine the contents of a disk without even having to take it over to their Amigas or emulators unless they are completely convinced to do so.
To be honest, I think many of these disks by being unfurled this way will create very enjoyable past-time browsing and reading just as they are here on the site. Pretty cool, right?
With all of this said, I currently have a measly 4 issues added so far. Once I've completed all of what I own of Volume 4 (1990) for Amiga Animation, I will likely expose it publicly. Then there are all of the other Volumes of that magazine I've bought and collected over the years (I don't own every single issue but I own a
lot and none of them are online), and all the other collections and disks I want to archive here. This will be a multi-year project but a very fun and meaningful one.
@JoeUser:
If you want to ship me any User Group disks for archiving, I can move those to the top of the queue and mail them back to you in a few weeks/months afterwards. I don't think UserGroup disks will take nearly as long to do since it eliminates the entire scanning/OCR element from the process. DM me and we can work it out.