Vore_Microcomputers/website
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add journaling_impl text (no img yet)

Browse source
This commit is contained in:
husky 2023-08-13 18:31:47 -07:00
parent 380e6058de
commit 05ef328bb4
No known key found for this signature in database
GPG key ID: 0FF8E73B15EAEC31
4 changed files with 128 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
blog/posts/journaling_impl/checklist.jpg Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 70 KiB

BIN
blog/posts/journaling_impl/files.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 6.1 KiB

122
blog/posts/journaling_impl/index.html Normal file
View file

@ -0,0 +1,122 @@
<html lang="en">
<head>
<title>vore microcomputers development blog - blog post title here</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<link rel="stylesheet" href="/microsoft.css"/>
<link rel="icon" type="image/x-icon" href="/favicon.ico">
</head>
<body>
<div class="container">
<div class="container-item">
<a href="/"><img src="/assets/logo_typeset.svg" alt="vore microcomputers logo"/></a>
<h1>VapUFS Implementation Hell: Week 2 of VoreStation Development</h1>
<h5>2023-08-13</h5>
<h4>written by Niko Chow-Stuart (Lead Developer)</h4>
<img src="/blog/posts/filesystems_and_apm/files.png" alt="drawing of the vap mascot in a drawer with two manila folders to the right"/>
<p>
while unfortunately, as i expected, i wasn't able to do as much work on the filesystem the week before last;
after managing to get somewhat back on track this week, i've finished nearly all aspects of VapUFS's
implementation!
<br/>
<br/>
regarding the work i've done, i should probably go more in-depth about this image that i posted on my
personal social media accounts a bit ago.
<img class="img-float-right" src="/blog/posts/journaling_impl/checklist.jpg" alt="checklist of things implemented for the filesystem, including direct access, creation and flushing of journal entries, and wrappers for journaled writes. the last entry, labeled directories, is unchecked."/>
<br/>
<br/>
starting at the section labeled "direct access", the implementation was started by writing some basic
functions for retrieving the core structures of the filesystem off of the disk (using a generic abstraction
layer for the disk, not with any actual disk drivers). this was an important step to take, as very early on,
i decided to make it as clear as possible that all structs would be stored on-disk in big-endian values.
because of this, i was able to add in a step in the retrieval process that (depending on architecture)
converts the big-endian values to the cpu's native byte-ordering, and back again when storing.
<br/>
<br/>
as the filesystem is journaled, i then began the work of implementing the creation of journal entries as
well as the somewhat more complex step of "flushing" the journal entries (i.e. performing them if they are
not corrupt, and then marking them as complete). this section of implementation took much longer than the
previous (which i'd say maybe took like an hour at most, including the time i spent procrastinating).
the source code (which i'll get to later in this post!) of the filesystem goes much more in-depth about
how everything works.
<br/>
<br/>
regarding journaling, there is one major aspect listed in the image, that being the classification of
single block writes and multi block writes. one thing that i believe i have yet to clarify in the source
code is that the defined disk block size for the filesystem should be <i>at least</i> large enough to be
able to store the largest struct for the file system (which i believe is the superblock, spanning around
156 bytes). this means that all metadata writes (which are journaled) can be performed using a single block.
i was originally planning on using this strategy for all writes, where they would be broken up into single
block writes in the journal; however, there is one area where this would be a bad idea: directories.
directories, in the typical UFS fashion, are stored in an inode's data blocks; however, they could
theoretically span multiple blocks in length. because of this, as well as the fact that they would be stored
in a potentially fragile b-tree structure, i decided to add a multi block journaled write. this way, if the
system is shut down during a directory contents update (e.g. new folder or file), the directory shouldn't
become corrupted.
<br/>
<br/>
another potential confusion in the original image would be the difference between "journaling" and
"journaled access" (which i admit is probably not the best way to label it). "journaling" here refers to
directly creating and executing journal entries, whereas "journaled access" refers to higher-level
functions that manage creating journaled writes for various metadata structs as well as file data. notably,
file data journaled writes will automatically decide between single block and multi block operations
depending on the amount of data being written. this section of implementation also took quite a bit, as it
required extra thought into the planning of journal executions and failure states (one of which i realised
while writing this that i forgot to implement, that being the deallocation of data blocks upon source block
corruption).
<br/>
<br/>
other than that, hopefully the only thing left is to write a serialisable b-tree implementation for
directory contents. i'm planning on also storing a pure array of file and directory names along-side the
b-tree in order to assist with recovery upon corruption, however none of that is written yet so i'll see
how things go when the time comes!
<br/>
<br/>
regarding the release of the VapUFS source code, it should be up by the time this goes public! and by that
i mean, you can view it here:
<a href="https://git.voremicrocomputers.com/voremicrocomputers/vap/vapfs/vapfs_ufs">link to git repo</a>.
if anyone has trouble viewing it, just shoot me an email at
<a href="mailto:nikocs@voremicrocomputers.com">nikocs@voremicrocomputers.com</a> and i'll try to fix it
asap!
<br/>
<br/>
that's it for the progress this week! hopefully i'll actually start releasing these weekly now that i have
a bit more time on my hands.
<br/>
i will note that donations to our LiberaPay are very much appreciated, especially as i'm currently bouncing
between this and some other stuff to try to pay for everything! :D
<script src="https://liberapay.com/voremicrocomputers/widgets/button.js"></script>
<noscript><a href="https://liberapay.com/voremicrocomputers/donate"><img alt="Donate using Liberapay" src="https://liberapay.com/assets/widgets/donate.svg"></a></noscript>
<br/>
other than that, have a good week and happy voring!
</p>
<hr/>
<p id="footer">
<br>contact us at <a href="mailto:nikocs@voremicrocomputers.com">nikocs@voremicrocomputers.com</a> <a
href="/pgp.asc">pgp key</a>
<br>
for abuse, copyright, or other legal issues, contact <a href="mailto:devnull@voremicrocomputers.com">devnull@voremicrocomputers.com</a><br>
<!-- or, call at <a href="tel:1-888-519-0437">+1 (888) 519-0437 (toll-free)</a> or <a href="tel:1-507-767-9433">+1 (507) RMS-YIFF (local)</a> -->
(our phone system is currently down, sorry!)
<br><br>
Vore Microcomputers is an unregistered trademark of Real Microsoft, LLC. all references to "the company"
are in reference to Real Microsoft, LLC and the name Vore Microcomputers is only a reference to the
hardware/software focused development project held under Real Microsoft, LLC. Real Microsoft, LLC is
in no way associated with the Microsoft Corporation or its products/projects.
<br><br>
<a href="https://climate.stripe.com/4MO1d9">Our Carbon Footprint</a>
</p>
</div>
<div class="container-item">
<div class="credit">
<img src="/xenia.png" alt="image of xenia, an anthropomorphic fox who was a contender for the linux mascot"/>
<span>image made by <a href="https://twitter.com/cathodegaytube/">@cathodegaytube</a>!</span>
</div>
</div>
</div>
</body>
</html>

6
microsoft.css
View file

@ -16,6 +16,12 @@ hr {
background-image: -o-linear-gradient(left, #f0f0f0, #8c8b8b, #f0f0f0);
}
.img-float-right {
float: right;
width: 40%;
padding: 10px;
}
.blog-listing {
margin-top: 100px;
margin-bottom: 100px;