I am happy to announce that my own operating system written from scratch JOS got some updates in recent months. Since the last update I was able to extend two essential capabilities to progress towards my big next goal (running a somewhat useful shell): Filesystem and Disk Driver. As usual, both are very rudimentary and incomplete. At the moment only read operations are possible, and even those are very limited. Still, I am relieved that this is working now. Both in combination enable me to drop the workaround that files have been stored directly in the binary of my OS.
As fileystem I went for Ext2, on older very simple filesystem which was (and is via its successors Ext3 and Ext4) in use by many if not most Linux systems. For the disk driver I am also going for on oldie: ATA. In the end, the code for both together is only approx 500 lines. I had actually procrastinated this for a while because I was awed by the expected complexity. While both caused some frustration loops in the end it clicked faster than expected.
I am even surprised that the performance is better than what I have expected from an unoptimized first working iteration. As you can see in the tracing view one exemplary file read syscall from userspace (still Doom 😉 ) is taking less than one 1 millisecond, and reading one sector (512 bytes) from the disk is taking 100 microseconds.
What actually has cost me way more time and nerves during the last months has been a refactoring without any clear purpose (and in hindsight probably could have been skipped): I enabled JOS to run with 4 KB pages. The page size is the fundamental unit used by the memory management unit of the CPU to organize memory. For historic reasons I have been using 2 MB pages. While 2 MB work fine, I guessed that this would lead too to less granularity. E.g. I was researching into IPC mechanisms, specifically via shared memory. And if each process has to share memory in 2 MB chunks the RAM quickly would be too full. So yeah, now its running with 4 KB pages size. Theoretically 2 MB still work and could be enabled as a compile time switch. However, I was too happy that 4 KB finally worked so I didnt find the mojo yet to test-and probably fix-2MB.
Speaking of testing: The automated tests I introduced in my last brief JOS update really paid off big time. I could check any change within a few seconds of automated tests and ensure nothing worsened. As in the end all above improvements didnt directly enable any new functionality, a lot improved under the hood and Doom is still running.