124 lines
8.4 KiB
HTML
124 lines
8.4 KiB
HTML
<html lang="en">
|
|
<head>
|
|
<title>vore microcomputers development blog - VapUFS Implementation Hell: Week 2 of VoreStation Development</title>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
|
|
<link rel="stylesheet" href="/microsoft.css"/>
|
|
<link rel="icon" type="image/x-icon" href="/favicon.ico">
|
|
</head>
|
|
<body>
|
|
<div class="container">
|
|
<div class="container-item">
|
|
<a href="/"><img src="/assets/logo_typeset.svg" alt="vore microcomputers logo"/></a>
|
|
|
|
<h1>VapUFS Implementation Hell: Week 2 of VoreStation Development</h1>
|
|
<h5>2023-08-13</h5>
|
|
<h4>written by Niko Chow-Stuart (Lead Developer)</h4>
|
|
|
|
<img src="/blog/posts/journaling_impl/journaling.png" alt="drawing of the vap mascot looking at a journal, with a pencil held up on their ear."/>
|
|
|
|
<p>
|
|
while unfortunately, as i expected, i wasn't able to do as much work on the filesystem the week before last;
|
|
after managing to get somewhat back on track this week, i've finished nearly all aspects of VapUFS's
|
|
implementation!
|
|
<br/>
|
|
<br/>
|
|
regarding the work i've done, i should probably go more in-depth about this image that i posted on my
|
|
personal social media accounts a bit ago.
|
|
<img class="img-float-right" src="/blog/posts/journaling_impl/checklist.jpg" alt="checklist of things implemented for the filesystem, including direct access, creation and flushing of journal entries, and wrappers for journaled writes. the last entry, labeled directories, is unchecked."/>
|
|
<br/>
|
|
<br/>
|
|
starting at the section labeled "direct access", the implementation was started by writing some basic
|
|
functions for retrieving the core structures of the filesystem off of the disk (using a generic abstraction
|
|
layer for the disk, not with any actual disk drivers). this was an important step to take, as very early on,
|
|
i decided to make it as clear as possible that all structs would be stored on-disk in big-endian values.
|
|
because of this, i was able to add in a step in the retrieval process that (depending on architecture)
|
|
converts the big-endian values to the cpu's native byte-ordering, and back again when storing.
|
|
<br/>
|
|
<br/>
|
|
as the filesystem is journaled, i then began the work of implementing the creation of journal entries as
|
|
well as the somewhat more complex step of "flushing" the journal entries (i.e. performing them if they are
|
|
not corrupt, and then marking them as complete). this section of implementation took much longer than the
|
|
previous (which i'd say maybe took like an hour at most, including the time i spent procrastinating).
|
|
the source code (which i'll get to later in this post!) of the filesystem goes much more in-depth about
|
|
how everything works.
|
|
<br/>
|
|
<br/>
|
|
regarding journaling, there is one major aspect listed in the image, that being the classification of
|
|
single block writes and multi block writes. one thing that i believe i have yet to clarify in the source
|
|
code is that the defined disk block size for the filesystem should be <i>at least</i> large enough to be
|
|
able to store the largest struct for the file system (which i believe is the superblock, spanning around
|
|
156 bytes). this means that all metadata writes (which are journaled) can be performed using a single block.
|
|
i was originally planning on using this strategy for all writes, where they would be broken up into single
|
|
block writes in the journal; however, there is one area where this would be a bad idea: directories.
|
|
directories, in the typical UFS fashion, are stored in an inode's data blocks; however, they could
|
|
theoretically span multiple blocks in length. because of this, as well as the fact that they would be stored
|
|
in a potentially fragile b-tree structure, i decided to add a multi block journaled write. this way, if the
|
|
system is shut down during a directory contents update (e.g. new folder or file), the directory shouldn't
|
|
become corrupted.
|
|
<br/>
|
|
<br/>
|
|
another potential confusion in the original image would be the difference between "journaling" and
|
|
"journaled access" (which i admit is probably not the best way to label it). "journaling" here refers to
|
|
directly creating and executing journal entries, whereas "journaled access" refers to higher-level
|
|
functions that manage creating journaled writes for various metadata structs as well as file data. notably,
|
|
file data journaled writes will automatically decide between single block and multi block operations
|
|
depending on the amount of data being written. this section of implementation also took quite a bit, as it
|
|
required extra thought into the planning of journal executions and failure states (one of which i realised
|
|
while writing this that i forgot to implement, that being the deallocation of data blocks upon source block
|
|
corruption).
|
|
<br/>
|
|
<br/>
|
|
other than that, hopefully the only thing left is to write a serialisable b-tree implementation for
|
|
directory contents. i'm planning on also storing a pure array of file and directory names along-side the
|
|
b-tree in order to assist with recovery upon corruption, however none of that is written yet so i'll see
|
|
how things go when the time comes!
|
|
<br/>
|
|
<br/>
|
|
regarding the release of the VapUFS source code, it should be up by the time this goes public! and by that
|
|
i mean, you can view it here:
|
|
<a href="https://git.voremicrocomputers.com/voremicrocomputers/vap/vapfs/vapfs_ufs">link to git repo</a>.
|
|
if anyone has trouble viewing it, just shoot me an email at
|
|
<a href="mailto:nikocs@voremicrocomputers.com">nikocs@voremicrocomputers.com</a> and i'll try to fix it
|
|
asap!
|
|
<br/>
|
|
<br/>
|
|
that's it for the progress this week! hopefully i'll actually start releasing these weekly now that i have
|
|
a bit more time on my hands.
|
|
<br/>
|
|
i will note that donations to our LiberaPay are very much appreciated, especially as i'm currently bouncing
|
|
between this and some other stuff to try to pay for everything! :D
|
|
<br/>
|
|
<script src="https://liberapay.com/voremicrocomputers/widgets/button.js"></script>
|
|
<noscript><a href="https://liberapay.com/voremicrocomputers/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a></noscript>
|
|
<br/>
|
|
<br/>
|
|
other than that, have a good week and happy voring!
|
|
</p>
|
|
|
|
<hr/>
|
|
<p id="footer">
|
|
<br>contact us at <a href="mailto:nikocs@voremicrocomputers.com">nikocs@voremicrocomputers.com</a> <a
|
|
href="/pgp.asc">pgp key</a>
|
|
<br>
|
|
for abuse, copyright, or other legal issues, contact <a href="mailto:devnull@voremicrocomputers.com">devnull@voremicrocomputers.com</a><br>
|
|
<!-- or, call at <a href="tel:1-888-519-0437">+1 (888) 519-0437 (toll-free)</a> or <a href="tel:1-507-767-9433">+1 (507) RMS-YIFF (local)</a> -->
|
|
(our phone system is currently down, sorry!)
|
|
<br><br>
|
|
Vore Microcomputers is an unregistered trademark of Real Microsoft, LLC. all references to "the company"
|
|
are in reference to Real Microsoft, LLC and the name Vore Microcomputers is only a reference to the
|
|
hardware/software focused development project held under Real Microsoft, LLC. Real Microsoft, LLC is
|
|
in no way associated with the Microsoft Corporation or its products/projects.
|
|
<br><br>
|
|
<a href="https://climate.stripe.com/4MO1d9">Our Carbon Footprint</a>
|
|
</p>
|
|
</div>
|
|
|
|
<div class="container-item">
|
|
<div class="credit">
|
|
<img src="/xenia.png" alt="image of xenia, an anthropomorphic fox who was a contender for the linux mascot"/>
|
|
<span>image made by <a href="https://twitter.com/cathodegaytube/">@cathodegaytube</a>!</span>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</body>
|
|
</html>
|