Implement basic FAT32 driver #2

ggnfs000 · January 09, 2017, 06:01:56 AM

Implement basic FAT32 driver #2

Since posting "Implement basic FAT32 driver" which should give me enough information to traverse through the file system to load any files, however I found one important of aspect has not been covered: File(s) with size more than one cluster size. For files with size less than 1 cluster size, the root directory entry is enough to get the content of that file. If more than that, we need FAT32.

http://www.pjrc.com/tech/8051/ide/fat32.html has very good simple coverage on this. From there, key information to dissects are:
- FAT32 data structure immediately follows the reserved sectors for given volume. On the VM I setup, I verified it by dumping the 1st sector of FAT32. To get the 1st sector of FAT32, used following formula:
Sector No. for volume + reserved sector = 800h+103ah=183ah from previous example. BTW, I deleted every files and created only 1 small text files test.txt and 1 large file python installable that has 20MB size. I will use 20MB as an example for file that has more than 1 cluster size.
- The first cluster No. is in the root directory content's file entry. However, subsequent cluster numbers are contained in the FAT32 structure.
To find it, first we note the 1st cluster No. from root directory content and use it to offset into FAT32's entry. FAT is a series of 32-bit cluster number's to help locating the clusters from the data area. Each offset from the start of FAT32 corresponds to the cluster No. and content of this offset is the next cluster No. The application should follow this linked list and grab those clusters from the disk (which is the actual file content) until it reaches the cluster No. FFFFFFF0 which signifies the end of file.

Dumping sector 800h+103ah resulted in following 1st sector of FAT32's content:

LBA 000000000000183A Size 00000200 bytes BlkIo BE5303A8
00000000: F8 FF FF 0F FF FF FF FF-FF FF FF 0F FF FF FF 0F *................*
00000010: FF FF FF 0F FF FF FF 0F-07 00 00 00 08 00 00 00 *................*
00000020: 09 00 00 00 0A 00 00 00-0B 00 00 00 0C 00 00 00 *................*
00000030: 0D 00 00 00 0E 00 00 00-0F 00 00 00 10 00 00 00 *................*
00000040: 11 00 00 00 12 00 00 00-13 00 00 00 14 00 00 00 *................*
00000050: 15 00 00 00 16 00 00 00-17 00 00 00 18 00 00 00 *................*
00000060: 19 00 00 00 1A 00 00 00-1B 00 00 00 1C 00 00 00 *................*
00000070: 1D 00 00 00 1E 00 00 00-1F 00 00 00 20 00 00 00 *............ ...*
00000080: 21 00 00 00 22 00 00 00-23 00 00 00 24 00 00 00 *!..."...#...$...*
00000090: 25 00 00 00 26 00 00 00-27 00 00 00 28 00 00 00 *%...&...'...(...*
000000A0: 29 00 00 00 2A 00 00 00-2B 00 00 00 2C 00 00 00 *)...*...+...,...*
000000B0: 2D 00 00 00 2E 00 00 00-2F 00 00 00 30 00 00 00 *-......./...0...*
000000C0: 31 00 00 00 32 00 00 00-33 00 00 00 34 00 00 00 *1...2...3...4...*
000000D0: 35 00 00 00 36 00 00 00-37 00 00 00 38 00 00 00 *5...6...7...8...*
000000E0: 39 00 00 00 3A 00 00 00-3B 00 00 00 3C 00 00 00 *9...:...;...<...*
000000F0: 3D 00 00 00 3E 00 00 00-3F 00 00 00 40 00 00 00 *=...>...?...@...*
00000100: 41 00 00 00 42 00 00 00-43 00 00 00 44 00 00 00 *A...B...C...D...*
00000110: 45 00 00 00 46 00 00 00-47 00 00 00 48 00 00 00 *E...F...G...H...*
00000120: 49 00 00 00 4A 00 00 00-4B 00 00 00 4C 00 00 00 *I...J...K...L...*
00000130: 4D 00 00 00 4E 00 00 00-4F 00 00 00 50 00 00 00 *M...N...O...P...*
00000140: 51 00 00 00 52 00 00 00-53 00 00 00 54 00 00 00 *Q...R...S...T...*
00000150: 55 00 00 00 56 00 00 00-57 00 00 00 58 00 00 00 *U...V...W...X...*
00000160: 59 00 00 00 5A 00 00 00-5B 00 00 00 5C 00 00 00 *Y...Z...[...\...*
00000170: 5D 00 00 00 5E 00 00 00-5F 00 00 00 60 00 00 00 *]...^..._...`...*
00000180: 61 00 00 00 62 00 00 00-63 00 00 00 64 00 00 00 *a...b...c...d...*
00000190: 65 00 00 00 66 00 00 00-67 00 00 00 68 00 00 00 *e...f...g...h...*
000001A0: 69 00 00 00 6A 00 00 00-6B 00 00 00 6C 00 00 00 *i...j...k...l...*
000001B0: 6D 00 00 00 6E 00 00 00-6F 00 00 00 70 00 00 00 *m...n...o...p...*
000001C0: 71 00 00 00 72 00 00 00-73 00 00 00 74 00 00 00 *q...r...s...t...*
000001D0: 75 00 00 00 76 00 00 00-77 00 00 00 78 00 00 00 *u...v...w...x...*
000001E0: 79 00 00 00 7A 00 00 00-7B 00 00 00 7C 00 00 00 *y...z...........*
000001F0: 7D 00 00 00 7E 00 00 00-7F 00 00 00 80 00 00 00 *................*

Let's look back at the root directory content. Test.rtf has file size of 01ch and python*.msi has file size of 013d0000h which corresponds to 28 and 20,774,912 bytes respectively. Starting cluster for each is 06 and 13d7h, respectively.

LBA 0000000000006800 Size 00000200 bytes BlkIo BE5303A8
00000000: 46 41 54 33 32 20 20 20-20 20 20 08 00 00 00 00 *FAT32 .....*
00000010: 00 00 00 00 00 00 C0 61-DA 48 00 00 00 00 00 00 *.......a.H......*
00000020: E5 20 00 44 00 6F 00 63-00 75 00 0F 00 31 6D 00 *. .D.o.c.u...1m.*
00000030: 65 00 6E 00 74 00 2E 00-72 00 00 00 74 00 66 00 *e.n.t...r...t.f.*
00000040: E5 4E 00 65 00 77 00 20-00 52 00 0F 00 31 69 00 *.N.e.w. .R...1i.*
00000050: 63 00 68 00 20 00 54 00-65 00 00 00 78 00 74 00 *c.h. .T.e...x.t.*
00000060: E5 45 57 52 49 43 7E 31-52 54 46 20 00 72 CB 61 *.EWRIC.1RTF .r.a*
00000070: DA 48 DA 48 00 00 CC 61-DA 48 03 00 07 00 00 00 *.H.H...a.H......*
00000080: 24 52 45 43 59 43 4C 45-42 49 4E 16 00 75 CB 61 *$RECYCLEBIN..u.a*
00000090: DA 48 DA 48 00 00 CC 61-DA 48 04 00 00 00 00 00 *.H.H...a.H......*
000000A0: E5 45 54 53 20 20 20 20-52 54 46 20 10 72 CB 61 *.ETS RTF .r.a*
000000B0: DA 48 DA 48 00 00 CC 61-DA 48 03 00 07 00 00 00 *.H.H...a.H......*
000000C0: 42 61 00 6D 00 64 00 36-00 34 00 0F 00 88 2E 00 *Ba.m.d.6.4......*
000000D0: 6D 00 73 00 69 00 00 00-FF FF 00 00 FF FF FF FF *m.s.i...........*
000000E0: 01 70 00 79 00 74 00 68-00 6F 00 0F 00 88 6E 00 *.p.y.t.h.o....n.*
000000F0: 2D 00 33 00 2E 00 33 00-2E 00 00 00 32 00 2E 00 *-.3...3.....2...*
00000100: 50 59 54 48 4F 4E 7E 31-4D 53 49 20 00 B6 D8 61 *PYTHON.1MSI ...a*
00000110: DA 48 DA 48 00 00 39 6D-5D 43 06 00 00 00 3D 01 *.H.H..9m]C....=.*
00000120: E5 6D 00 65 00 6E 00 74-00 2E 00 0F 00 9F 74 00 *.m.e.n.t......t.*
00000130: 78 00 74 00 00 00 FF FF-FF FF 00 00 FF FF FF FF *x.t.............*
00000140: E5 4E 00 65 00 77 00 20-00 54 00 0F 00 9F 65 00 *.N.e.w. .T....e.*
00000150: 78 00 74 00 20 00 44 00-6F 00 00 00 63 00 75 00 *x.t. .D.o...c.u.*
00000160: E5 45 57 54 45 58 7E 31-54 58 54 20 00 6F 4E 82 *.EWTEX.1TXT .oN.*
00000170: DA 48 DA 48 00 00 4F 82-DA 48 00 00 00 00 00 00 *.H.H..O..H......*
00000180: 54 45 53 54 20 20 20 20-54 58 54 20 10 6F 4E 82 *TEST TXT .oN.*
00000190: DA 48 DA 48 00 00 59 82-DA 48 D7 13 1C 00 00 00 *.H.H..Y..H......*
000001A0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*
000001B0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*
000001C0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*
000001D0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*
000001E0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*
000001F0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

To confirm these numbers, first, lets list the file content to confirm the size:

Directory of: fs1:\

10/29/13 01:41p 20,774,912 python-3.3.2.amd64.msi

06/26/16 04:18p 28 TEST.txt

2 File(s) 20,774,940 bytes

0 Dir(s)

To confirm the file cluster numbers, let's dump first sector of 1st cluster for each of these files. For larger than 1 sector file (python installed), will dump several of the clusters:

For python installer, we can see sector 06 from root directory entry. In FAT, this location has link sector 07 which has link to 08 and so on therefore its content appears to have been written quite sequentially.

LBA 000000000000183A Size 00000200 bytes BlkIo BE5303A8
00000000: F8 FF FF 0F FF FF FF FF-FF FF FF 0F FF FF FF 0F *................*
00000010: FF FF FF 0F FF FF FF 0F-07 00 00 00 08 00 00 00 *................*
00000020: 09 00 00 00 0A 00 00 00-0B 00 00 00 0C 00 00 00 *................*
00000030: 0D 00 00 00 0E 00 00 00-0F 00 00 00 10 00 00 00 *................*

Therefore first sector of first few cluster occupied by python installer's:

(06 - 2) * 8 + 6800h = 6820h

(07 - 2) * 8 + 6800h = 6828h

For test.txt file since it has been written later after deleting couple of files and it is also written after python installer, the cluster is way to later, therefore we need to find the sector number and dump again using the formula we used:

Sector No. - 2 * 8 + 1st sector of vol:

(13 d7 - 2) * 8 + 6800h = 106a8h.

Sector 106a8 dump content was as follows, which confirms we found the correct sector for test.txt file. And this is the only sector in the only cluster that test.txt occupies.

LBA 00000000000106A8 Size 00000200 bytes BlkIo BE5303A8

00000000: 54 48 49 53 20 49 53 20-41 20 54 45 53 54 20 46 *THIS IS A TEST F*

00000010: 49 4C 45 20 43 4F 4E 54-45 4E 54 2E 00 00 00 00 *ILE CONTENT.....*

00000020: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000030: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000040: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000050: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000060: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000070: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000080: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000090: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000000A0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000000B0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000000C0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000000D0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000000E0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000000F0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000100: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000110: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000120: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000130: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000140: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000150: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000160: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000170: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000180: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

00000190: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000001A0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000001B0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000001C0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000001D0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000001E0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

000001F0: 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 *................*

Sector 6820h is the cluster 06's first sector and sector 6828h is the cluster 07's first sector, both occupied by python installer. To verify the content we also dump those sector's and compare with the actual file:

sector 6820h:

LBA 0000000000006820 Size 00000200 bytes BlkIo BE5303A8

00000000: D0 CF 11 E0 A1 B1 1A E1-00 00 00 00 00 00 00 00 *................*

00000010: 00 00 00 00 00 00 00 00-3E 00 04 00 FE FF 0C 00 *........>.......*

00000020: 06 00 00 00 00 00 00 00-02 00 00 00 05 00 00 00 *................*

00000030: 01 00 00 00 00 00 00 00-00 10 00 00 09 00 00 00 *................*

00000040: 01 00 00 00 FE FF FF FF-00 00 00 00 00 00 00 00 *................*

00000050: 5F 00 00 00 6F 00 00 00-70 00 00 00 71 00 00 00 *_...o...p...q...*

00000060: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000070: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000080: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000090: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000000A0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000000B0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000000C0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000000D0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000000E0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000000F0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000100: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000110: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000120: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000130: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000140: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000150: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000160: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000170: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000180: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

00000190: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000001A0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000001B0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000001C0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000001D0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000001E0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

000001F0: FF FF FF FF FF FF FF FF-FF FF FF FF FF FF FF FF *................*

sector 6828h:

LBA 0000000000006828 Size 00000200 bytes BlkIo BE5303A8

00000000: FD FF FF FF 0F 00 00 00-03 00 00 00 04 00 00 00 *................*

00000010: 05 00 00 00 06 00 00 00-07 00 00 00 08 00 00 00 *................*

00000020: 0C 00 00 00 FE FF FF FF-0B 00 00 00 15 00 00 00 *................*

00000030: 0D 00 00 00 0E 00 00 00-10 00 00 00 FE FF FF FF *................*

00000040: 11 00 00 00 12 00 00 00-13 00 00 00 14 00 00 00 *................*

00000050: 4C 00 00 00 41 13 00 00-17 00 00 00 18 00 00 00 *L...A...........*

00000060: 19 00 00 00 1A 00 00 00-1B 00 00 00 1C 00 00 00 *................*

00000070: 1D 00 00 00 1E 00 00 00-1F 00 00 00 FE FF FF FF *................*

00000080: 21 00 00 00 22 00 00 00-23 00 00 00 24 00 00 00 *!..."...#...$...*

00000090: 25 00 00 00 26 00 00 00-27 00 00 00 28 00 00 00 *%...&...'...(...*

000000A0: 29 00 00 00 2A 00 00 00-2B 00 00 00 2C 00 00 00 *)...*...+...,...*

000000B0: 2D 00 00 00 2E 00 00 00-2F 00 00 00 30 00 00 00 *-......./...0...*

000000C0: 31 00 00 00 32 00 00 00-33 00 00 00 34 00 00 00 *1...2...3...4...*

000000D0: 35 00 00 00 36 00 00 00-37 00 00 00 FE FF FF FF *5...6...7.......*

000000E0: 39 00 00 00 3A 00 00 00-3B 00 00 00 3C 00 00 00 *9...:...;...<...*

000000F0: FE FF FF FF 3E 00 00 00-3F 00 00 00 40 00 00 00 *....>...?...@...*

00000100: 41 00 00 00 42 00 00 00-43 00 00 00 44 00 00 00 *A...B...C...D...*

00000110: 45 00 00 00 46 00 00 00-47 00 00 00 48 00 00 00 *E...F...G...H...*

00000120: 49 00 00 00 4A 00 00 00-4B 00 00 00 FE FF FF FF *I...J...K.......*

00000130: 4D 00 00 00 4E 00 00 00-4F 00 00 00 50 00 00 00 *M...N...O...P...*

00000140: 51 00 00 00 52 00 00 00-53 00 00 00 54 00 00 00 *Q...R...S...T...*

00000150: 55 00 00 00 56 00 00 00-57 00 00 00 58 00 00 00 *U...V...W...X...*

00000160: 59 00 00 00 5A 00 00 00-5B 00 00 00 5C 00 00 00 *Y...Z...[...\...*

00000170: 5D 00 00 00 5E 00 00 00-60 00 00 00 FD FF FF FF *]...^...`.......*

00000180: 61 00 00 00 62 00 00 00-63 00 00 00 64 00 00 00 *a...b...c...d...*

00000190: FE FF FF FF 66 00 00 00-67 00 00 00 68 00 00 00 *....f...g...h...*

000001A0: 69 00 00 00 6A 00 00 00-6B 00 00 00 6C 00 00 00 *i...j...k...l...*

000001B0: FE FF FF FF 6E 00 00 00-33 13 00 00 FD FF FF FF *....n...3.......*

000001C0: FD FF FF FF FD FF FF FF-73 00 00 00 74 00 00 00 *........s...t...*

000001D0: 75 00 00 00 76 00 00 00-77 00 00 00 78 00 00 00 *u...v...w...x...*

000001E0: 79 00 00 00 7A 00 00 00-7B 00 00 00 7C 00 00 00 *y...z...........*

000001F0: 7D 00 00 00 7E 00 00 00-7F 00 00 00 80 00 00 00 *................*

Without saying much, I opened the python installer in Dos Navigator's binary viewer and looked at offset 0000h which corresponds to cluster 06h/sector 0h and offset 1000h which corresponds to cluster 07h/sector 0h. The offset 1000h corresponds to cluster worth of data (4K or 8 sector) with the current file system's setup and since pretty much all (if not most) of python installer's cluster is sequentially.

Source: Implement basic FAT32 driver #2

Implement basic FAT32 driver #2

ggnfs000