![BeCor[o]ner](http://reveng.110mb.com/images/becoroner.gif)
04/2004 by Zadig.
Tools needed:
Introduction:
This document is intended to explain the fix on the 1GB memory limitation on
BeOS r5 based systems (pe/pro 5.xx, dano/phos/zeta...). The issue is the
following: If your computer has too much memory, beos will not boot but reboot
at early stage.
Discussions on this can be found at several places on the net. This is a known
problem for times and would have been already fixed if be inc were still
alive.
This document will explain, with more or less details the 3 methods that has
been used to make beos boot on such systems:
To understand the exact origin of the issue we must know a minimum part of the virtual memory managment on BeOS, and especially how is mapped a process on memory. Based on different discussions found on the net, here is a drawing showing how this is done:
_____________
Physical | |
space addr | RAM |
|_____________|
. .
. .
==============================================================
. .
. .
_______________________ _______________________
Logical | | || | |
space addr | 1 | 2 || 3 | 4 | Process 1
|_____________|_________||_______|_______________|
. .
. . _______________________
//////////////////////// | | |
//////////////////////// | 3 | 4 | Process 2
//////////////////////// |_____|_________________|
. .
. . _______________________
//////////////////////// | | |
//////////////////////// | 3 | 4 | Process n
//////////////////////// |___________|___________|
0 2GB 4GB
\__________ __________/ \__________ __________/
\/ \/
Kernel space User space
The logical addr space of each process is separated in 2 areas:
Such a design implementation allow some simplifications in the virtual memory managment. Win2k uses a similar approach. By the way the separation in 2 areas, brings a first limit: Each process can access at best 2GB of data. Moreover each area is also composed of several sections:
This is all we need to know about this. More on virtual memory managment can be found in [OSIDP].
The consequence of the structure of the kernel space is the following: The
bigger is section 1, the smaller is section 2. If section 1 is too big, there is
not enough space to hold the necessary datas in section 2 and the os can not
boot. This is what happens when we use ~1Gb of memory.
To avoid this problem there are several solutions:
Obviously, we choosed the third solution: This can be done by fooling the part that scans the size of the memory. This will make BeOS boot but a part of the memory will not be available. Doing this was tried at 3 different places: The kernel, the loader and the bios.
Let's start with the first option that we have to make this patch. Disasm the kernel with reveng, and search for interesting references. here is what euan found:
Function compute_memsize Referenced at Address(es): 0005EBE4 0005fcd8: 55 push %ebp 0005fcd9: 89e5 mov %esp,%ebp 0005fcdb: 53 push %ebx Reference to string "memmap" 0005fcdc: 6880eb0700 push $0x7eb80 Reference to function "get_boot_item" 0005fce1: e86ea5ffff call 5a254 0005fce6: 83c404 add $0x4,%esp 0005fce9: 85c0 test %eax,%eax ; if memmap found 0005fceb: 7513 jne 5fd00 ; jump to analyse it Reference to string "memsize" 0005fced: 6887eb0700 push $0x7eb87 Reference to function "get_boot_item" 0005fcf2: e85da5ffff call 5a254 0005fcf7: 8b00 mov (%eax),%eax 0005fcf9: eb40 jmp 5fd3b 0005fcfb: 90 nop 0005fcfc: 8d742600 lea 0x0(%esi,1),%esi Referenced by (conditionnal) jump(s) at Address(es): 0005FCEB 0005fd00: 31db xor %ebx,%ebx ; ebx = 0 0005fd02: 31c9 xor %ecx,%ecx ; ecx = 0 0005fd04: 83780400 cmpl $0x0,0x4(%eax) ; if(memmap[0].size == 0) 0005fd08: 7420 je 5fd2a ; jump label1 0005fd0a: 8db600000000 lea 0x0(%esi),%esi Referenced by (conditionnal) jump(s) at Address(es): 0005FD28 0005fd10: 8b54c804 mov 0x4(%eax,%ecx,8),%edx ; edx = memmap[ecx * sizeof(memmap_entry)].size 0005fd14: 0314c8 add (%eax,%ecx,8),%edx ; edx += memmap[ecx * sizeof(memmap_entry)].offset 0005fd17: 39da cmp %ebx,%edx ; if(edx < ebx) 0005fd19: 7202 jb 5fd1d ; jump label2 0005fd1b: 89d3 mov %edx,%ebx ; ebx = edx Referenced by (conditionnal) jump(s) at Address(es): 0005FD19 0005fd1d: 41 inc %ecx ; ecx++ 0005fd1e: 83f903 cmp $0x3,%ecx ; if(ecx > 3) 0005fd21: 7f07 jg 5fd2a ; jump label1 0005fd23: 837cc80400 cmpl $0x0,0x4(%eax,%ecx,8) ; if( memmap[ecx * sizeof(memmap_entry)].size != 0) 0005fd28: 75e6 jne 5fd10 ; jump label3 Referenced by (conditionnal) jump(s) at Address(es): 0005FD08 0005FD21 0005fd2a: 891da4fe0900 mov %ebx,0x9fea4 ; actual_memsize = ebx 0005fd30: 8d83ffff3f00 lea 0x3fffff(%ebx),%eax ; eax = actual_memsize + 0x3fffff 0005fd36: 250000c0ff and $0xffc00000,%eax ; eax &= 0xffc00000 Referenced by (conditionnal) jump(s) at Address(es): 0005FCF9 0005fd3b: a37c700a00 mov %eax,0xa707c 0005fd40: 8b5dfc mov 0xfffffffc(%ebp),%ebx 0005fd43: 89ec mov %ebp,%esp 0005fd45: 5d pop %ebp 0005fd46: c3 ret
At this point it seems that the kernel gets the physical size from the loader (via get_boot_item), and uses this value to do anything related to memory managment. If the kernel only uses "compute_memsize" to know the size and mapping of the memory, we will just have to patch this function. If there are several places where the loader is involved, it will be safer to do the patch in the loader.
A second point is the memmap array. At 0x5fd1e we can see that it has 4 entries, but what is the structure of these entries? mmu_man gave us it:
struct memmap_entry
{
uint32 base;
uint32 extend;
};
memmap_entry memmap[4];
He used this structure in one of his tools: sysinfo. It is a driver that dumps some variables and allow to print them on a terminal. Very usefull to get system informations! When we use it to dump the memmap array, we find the following values for a 512MB ram system:
memmap[4] =
{
{0, 654336},
{1048576, 535756800},
{0, 0},
{0, 0}
}
It seems that the first entry is the same on any configuration. So I tried to reduce the extend field of the second entry. Here is the patch:
Reference to string "memmap" 0005fcdc: 6880eb0700 push $0x7eb80 Reference to function "get_boot_item" 0005fce1: e86ea5ffff call 5a254 0005fce6: 83c404 add $0x4,%esp /*--------- patch code ---------------*/ 0005fce9: 8d580c lea 0xc(%eax),%ebx 0005fcec: b90000ef07 mov $0x7ef0000,%ecx 0005fcf1: 890b mov %ecx,(%ebx) 0005fcf3: eb0b jmp 5fd00 /*--------- end patch code -----------*/ 0005fd2a: 891da4fe0900 mov %ebx,0x9fea4 0005fd30: 8d83ffff3f00 lea 0x3fffff(%ebx),%eax 0005fd36: 250000c0ff and $0xffc00000,%eax 0005fd3b: a37c700a00 mov %eax,0xa707c 0005fd40: 8b5dfc mov 0xfffffffc(%ebp),%ebx 0005fd43: 89ec mov %ebp,%esp 0005fd45: 5d pop %ebp 0005fd46: c3 retThe kernel goes much deeper but crashes when mounting the boot partition. This is a great step because it is one of the last things that is done. Here is the crashlog that euan had on his 1GB machine:
Sysinit2:12 Flo_init : cyl_map[0].address = 0x1000 size = 48000 Flo_init : cyl_map[1].address = 0x0 size = 0 Floppy uninit() Fmap could not find disk with BIOS id 81 Flo_init : cyl_map[0].address = 0x1000 size = 48000 Flo_init : cyl_map[1].address = 0x0 size = 0 Floppy uninit() Fmap could not find disk with BIOS id 81 Could not open /dev/disk/virtual/fmap/0/0_0 type KERNEL PANIC : was unable to mount /dev/disk/virtual/fmap/0/0_0 type bfs on /boot
The crash occurs in "mount_volumes". It seems that the kernel did not find the
boot disk from the ide devices list. Disabling bios calls in the kernel does not
change anything.
Mmu_man had one idea about this: "the memory mapped devices
from the PCI bus [may] colide with existing RAM which isn't recognized." If it
what is happening, we will have to know much more about the memory management of
beos (at least more than what I know ;)). The function "init_io" has a lot of
references to "memsize". This may be where things are going wrong...
This is where the kernel story ends. I did not have time (and probably the necessary knowledge) to continue on this. Maybe this crash was just due to libroot that is also getting memsize from the loader. I do not know.
Patching the loader will allow to completely fool the kernel. This is a correct
place to do a patch. The first things that we need is a set of tools to
disassemble/generate the loader. This was done some times ago by pirge to
customize the boot pictures. I recommend that you start by reading [PIRGE] . Based on his loader extractor, I wrote 3 tools that I
use to manipulate the loader.
using loader_extract, you get 3 files: "boot.img", "beos", and "images". "boot.img" is the first part of the loader. It uncompresses the 2 other parts. "images" contains the boot images that are displayed (including the boot icons for example). "beos" is the part that interests us. It contains the main code of the loader. Since this code is raw asm, reveng can not be used to disassemble it. To do this, ndisasm (provided with nasm) will do the trick: ndisasm -u beos > beos.asm. The "u" option tells nasm that we use 32bit asm (default is 16bit).
So now we must find the interesting part inside 115kB of raw asm! To find it, we must know how to get the memory size from the bios. There are several ways to do this. Here are the usefull bios calls descriptions from [BROWN]:
int15, GET SYSTEM MEMORY MAP:
AX = E820h Return: CF clear if successful EAX = 534D4150h ('SMAP') ES:DI buffer filled EBX = next offset from which to copy or 00000000h if all done ECX = actual length returned in bytes CF set on error AH = error code (86h) (see #00496 at INT 15/AH=80h) Notes: Originally introduced with the Phoenix BIOS v4.0, this function is now supported by most newer BIOSes, since various versions of Windows call it to find out about the system memory. A maximum of 20 bytes will be transferred at one time, even if ECX is higher; some BIOSes (e.g. Award Modular BIOS v4.50PG) ignore the value of ECX on entry, and always copy 20 bytes. Some BIOSes expect the high word of EAX to be clear on entry, i.e. EAX=0000E820h. If this function is not supported, an application should fall back to AX=E802h, AX=E801h, and then AH=88h. The BIOS is permitted to return a nonzero continuation value in EBX and indicate that the end of the list has already been reached by returning with CF set on the next iteration. This function will return base memory and ISA/PCI memory contiguous with base memory as normal memory ranges; it will indicate chipset-defined address holes which are not in use and motherboard memory-mapped devices, and all occurrences of the system BIOS as reserved; standard PC address ranges will not be reported
Format of Phoenix BIOS system memory map address range descriptor:
(Table 00581) Values for System Memory Map address type: |
int15, GET MEMORY SIZE FOR >64M CONFIGURATIONS:AX = E801h Return: CF clear if successful AX = extended memory between 1M and 16M, in K (max 3C00h = 15MB) BX = extended memory above 16M, in 64K blocks CX = configured memory 1M to 16M, in K DX = configured memory above 16M, in 64K blocks CF set on error. Notes: Supported by the A03 level (6/14/94) and later XPS P90 BIOSes, as well as the Compaq Contura, 3/8/93 DESKPRO/i, and 7/26/93 LTE Lite 386 ROM BIOS. Supported by AMI BIOSes dated 8/23/94 or later. On some systems, the BIOS returns AX=BX=0000h; in this case, use CX and DX instead of AX and BX. This interface is used by Windows NT 3.1, OS/2 v2.11/2.20, and is used as a fall-back by newer versions if AX=E820h is not supported. This function is not used by MS-DOS 6.0 HIMEM.SYS when an EISA machine (for example with parameter /EISA) (see also MEM F000h:FFD9h), or no Compaq machine was detected, or parameter /NOABOVE16 was given. |
int15, GET EXTENDED MEMORY SIZE (286+):AH = 88h Return: CF clear if successful AX = number of contiguous KB starting at absolute address 100000h CF set on error AH = status 80h invalid command (PC,PCjr) 86h unsupported function (XT,PS30) Notes: TSRs which wish to allocate extended memory to themselves often hook this call, and return a reduced memory size. They are then free to use the memory between the new and old sizes at will.. The standard BIOS only returns memory between 1MB and 16MB; use AH=C7h for memory beyond 16MB. Not all BIOSes correctly return the carry flag, making this call unreliable unless one first checks whether it is supported through a mechanism other than calling the function and testing CF. Due to applications not dealing with more than 24-bit descriptors (286), Windows 3.0 has problems when this function reports more than 15 MB. Some releases of HIMEM.SYS are therefore limited to use only 15 MB, even when this function reports more. |
The most complete function is the first one (ax = 0xe820). Searching in the loader asm deadlisting gives a very interesting result at offset 0x683. Let's look at it...
Looking around this "e820" reference we see a lot of interesting things. The function at that offset begins at offset 0x5d0 and ends at 0x87a. Let's study this whole code:
Function "GetMemoryMapping":
000005D0 83EC30 sub esp,byte +0x30
000005D3 55 push ebp
000005D4 57 push edi
000005D5 56 push esi
000005D6 53 push ebx
000005D7 E800000000 call 0x5dc ; this is certainly generated by gcc...
000005DC 5B pop ebx
000005DD 81C39CC00100 add ebx,0x1c09c ; Yes, I'm sure it is => coded in C
; => gop = 0x1c678
000005E3 8B542448 mov edx,[esp+0x48]
000005E7 8D04D500000000 lea eax,[edx*8+0x0]
000005EE 50 push eax
000005EF 6A00 push byte +0x0
000005F1 8B4C244C mov ecx,[esp+0x4c]
000005F5 51 push ecx
000005F6 E8E51A0100 call 0x120e0
000005FB 8B830C000000 mov eax,[ebx+0xc]
00000601 C744242C00000000 mov dword [esp+0x2c],0x0
00000609 31F6 xor esi,esi ; esi = 0
0000060B 31ED xor ebp,ebp ; ebp = 0
0000060D 8B00 mov eax,[eax]
0000060F 0500100000 add eax,0x1000
00000614 89442428 mov [esp+0x28],eax
00000618 83C40C add esp,byte +0xc
0000061B EB62 jmp short 0x67f
This is the start of the function. Few things are done here: Some space is reserved in the stack, some initializations are done . One interesting point here is the call at 0x5d7. This is typically generated by gcc. So this means that it was probably written in C and not asm.
First_Method: (ax=e820) 0000061D 8B442430 mov eax,[esp+0x30] 00000621 8B4C2428 mov ecx,[esp+0x28] 00000625 89442420 mov [esp+0x20],eax 00000629 83F914 cmp ecx,byte +0x14 0000062C 7539 jnz 0x667 0000062E 8B54241C mov edx,[esp+0x1c] ; edx = base_buffer 00000632 8B4210 mov eax,[edx+0x10] 00000635 83F801 cmp eax,byte +0x1 ; if not usable mem 00000638 752D jnz 0x667 ; jump 0000063A 8B02 mov eax,[edx] ;\ 0000063C 8B5204 mov edx,[edx+0x4] ; \ 0000063F 8B542444 mov edx,[esp+0x44] ; - Get offset 00000643 8904B2 mov [edx+esi*4],eax ; / 00000646 8B54241C mov edx,[esp+0x1c] ;/ 0000064A 46 inc esi ;\ 0000064B 8B4208 mov eax,[edx+0x8] ; \ 0000064E 8B520C mov edx,[edx+0xc] ; - Get Size 00000651 8B542444 mov edx,[esp+0x44] ; / 00000655 8904B2 mov [edx+esi*4],eax ;/ 00000658 46 inc esi ;\ 00000659 8B44B2FC mov eax,[edx+esi*4-0x4] ; - eax = addr max 0000065D 0344B2F8 add eax,[edx+esi*4-0x8] ;/ 00000661 39E8 cmp eax,ebp ;\ 00000663 7602 jna 0x667 ; - Set ebp to highest mapped mem addr 00000665 89C5 mov ebp,eax ;/ 00000667 837C242000 cmp dword [esp+0x20],byte +0x0 ; all memory scaned ? 0000066C 7451 jz 0x6bf ; jump end scan 0000066E 83F914 cmp ecx,byte +0x14 ; did we get 20 byte 00000671 754C jnz 0x6bf ; if not, jump end scan 00000673 8B4C2448 mov ecx,[esp+0x48] ; ecx = max map entry num 00000677 89C8 mov eax,ecx ;\ 00000679 01C8 add eax,ecx ; - ecx *= 2 0000067B 39C6 cmp esi,eax 0000067D 7340 jnc 0x6bf ; end if to much entry read 0000067F 8B442420 mov eax,[esp+0x20] 00000683 C744242420E80000 mov dword [esp+0x24],0xe820 0000068B 89442430 mov [esp+0x30],eax 0000068F C744242814000000 mov dword [esp+0x28],0x14 ; buffer length = 20 00000697 C744242C50414D53 mov dword [esp+0x2c],0x534d4150 ; 'SMAP' 0000069F C744243C00100000 mov dword [esp+0x3c],0x1000 000006A7 8D7C2424 lea edi,[esp+0x24] 000006AB 57 push edi 000006AC 57 push edi 000006AD 6A15 push byte +0x15 000006AF E864F9FFFF call 0x18 ; call int 15 eax e820 000006B4 83C40C add esp,byte +0xc 000006B7 A801 test al,0x1 000006B9 0F845EFFFFFF jz near 0x61d 000006BF 81FDFFFF3F00 cmp ebp,0x3fffff ; memory length >= 4MB ? 000006C5 7704 ja 0x6cb ; Jump 000006C7 31F6 xor esi,esi ; esi = 0 000006C9 EB18 jmp short 0x6e3 ; jump fallback mapping 000006CB 89D8 mov eax,ebx 000006CD 056898FFFF add eax,0xffff9868 ; eax = ebx - 26520 000006D2 50 push eax 000006D3 E852CE0000 call 0xd52a 000006D8 83C404 add esp,byte +0x4 000006DB 85F6 test esi,esi 000006DD 0F8566010000 jnz near 0x849
This is where interesting things are. Comments are self explainatory I think. Everything begins at 0x67f (jumped from 0x61b). From here all params are filled to call int15. Depending on the result (if the call succeeds, and the map entry is valid), the memmap entry is copied in an array and the physical memory size is saved in ebp. At the end of the loop, if all went ok and the memory size is big enough we use the settings we found. On the other way, we jump to a first fallback method. We can also see here that the function seems to have 2 params: A pointer to the memap array to get, and the max number of entries to read. These 2 params are used at 0x673 and 0x63f for example. Now let's see what is the first fallback method:
Fallback_Method1: (ax=e801) 000006E3 57 push edi 000006E4 57 push edi 000006E5 6A12 push byte +0x12 ; int 12 : getmemorysize in kbyte 000006E7 E82CF9FFFF call 0x18 000006EC 8B542450 mov edx,[esp+0x50] 000006F0 C704B200000000 mov dword [edx+esi*4],0x0 000006F7 31C0 xor eax,eax 000006F9 668B442430 mov ax,[esp+0x30] 000006FE 46 inc esi 000006FF C1E00A shl eax,0xa ; eax *= 1024 00000702 8904B2 mov [edx+esi*4],eax 00000705 C744243001E80000 mov dword [esp+0x30],0xe801 0000070D 57 push edi 0000070E 57 push edi 0000070F 46 inc esi 00000710 6A15 push byte +0x15 ; int 15 00000712 E801F9FFFF call 0x18 00000717 83C418 add esp,byte +0x18 0000071A A801 test al,0x1 ; function not supported ? 0000071C 755A jnz 0x778 ; then fallback 2 0000071E 66837C242800 cmp word [esp+0x28],byte +0x0 00000724 750A jnz 0x730 00000726 66837C242C00 cmp word [esp+0x2c],byte +0x0 0000072C 744A jz 0x778 0000072E EB1A jmp short 0x74a 00000730 8B4C2444 mov ecx,[esp+0x44] 00000734 C704B100001000 mov dword [ecx+esi*4],0x100000 0000073B 31C0 xor eax,eax 0000073D 668B442428 mov ax,[esp+0x28] 00000742 46 inc esi 00000743 C1E00A shl eax,0xa 00000746 8904B1 mov [ecx+esi*4],eax 00000749 46 inc esi 0000074A 66837C242C00 cmp word [esp+0x2c],byte +0x0 00000750 741D jz 0x76f 00000752 8B442444 mov eax,[esp+0x44] 00000756 C704B000000001 mov dword [eax+esi*4],0x1000000 0000075D 31C0 xor eax,eax 0000075F 668B44242C mov ax,[esp+0x2c] 00000764 8B542444 mov edx,[esp+0x44] 00000768 C1E010 shl eax,0x10 ; eax *= 64k 0000076B 8944B204 mov [edx+esi*4+0x4],eax 0000076F 89D8 mov eax,ebx 00000771 058598FFFF add eax,0xffff9885 00000776 EB3E jmp short 0x7b6
This time the loader uses the int15 ax=e801 bios call. This returns the total amount of memory, not a detailed mapping. Once again if this call fails, we jump to a fallback method:
Fallback_Method2: (ah=88) 00000778 C744242400880000 mov dword [esp+0x24],0x8800 00000780 57 push edi 00000781 57 push edi 00000782 6A15 push byte +0x15 00000784 E88FF8FFFF call 0x18 00000789 83C40C add esp,byte +0xc 0000078C 8D7E01 lea edi,[esi+0x1] 0000078F 66837C242400 cmp word [esp+0x24],byte +0x0 ; function not supported ? 00000795 742D jz 0x7c4 ; then fallback 3 00000797 8B4C2444 mov ecx,[esp+0x44] 0000079B C704B100001000 mov dword [ecx+esi*4],0x100000 000007A2 31C0 xor eax,eax 000007A4 668B442424 mov ax,[esp+0x24] 000007A9 C1E00A shl eax,0xa 000007AC 8904B9 mov [ecx+edi*4],eax 000007AF 89D8 mov eax,ebx 000007B1 05A298FFFF add eax,0xffff98a2 000007B6 50 push eax 000007B7 E86ECD0000 call 0xd52a 000007BC 83C404 add esp,byte +0x4 000007BF E985000000 jmp 0x849
Here the int15 ax=88 call is used. This returns the size of the memory in ax, in 1kB chunks. This means that it works only with up to 64MB ram. This is used for old processors. Finally, if this does not work (When installing BeOS on a dinausor I suppose), a third fallback method is used. I will not detail it here because I am not sure of what it is. It may be a default mapping code. Anyway, if one of the methods worked, we jump to the last part of the function:
Mapping_found: 00000849 8B4C2448 mov ecx,[esp+0x48] 0000084D 31ED xor ebp,ebp ; ebp = 0 0000084F 31F6 xor esi,esi ; esi = 0 00000851 89CA mov edx,ecx ;\ 00000853 01CA add edx,ecx ; - edx = max map entry nb *2 00000855 39D5 cmp ebp,edx 00000857 7318 jnc 0x871 ; jump if equal 00000859 8B4C2444 mov ecx,[esp+0x44] ; ecx = base map addr 0000085D 8B44B104 mov eax,[ecx+esi*4+0x4] ;\ 00000861 0304B1 add eax,[ecx+esi*4] ; - eax = cur entry max addr 00000864 39E8 cmp eax,ebp ;\ 00000866 7602 jna 0x86a ; - ebp = max addr 00000868 89C5 mov ebp,eax ;/ 0000086A 83C602 add esi,byte +0x2 0000086D 39D6 cmp esi,edx 0000086F 72E8 jc 0x859 ; jump if diff 00000871 89E8 mov eax,ebp ; return max map addr 00000873 5B pop ebx 00000874 5E pop esi 00000875 5F pop edi 00000876 5D pop ebp 00000877 83C430 add esp,byte +0x30 0000087A C3 ret
At last here is the end! There are not much things done here. The max mapped address is calculated once again and is the return code of the function. Now we have everything we need to think about a patch: This function gets the memory map of the pc and the total memory size, thus it is certainly the initial place where all memory informations are get. This the function that has to be patched.
We are now almost sure that we found the place to patch. To be sure of this there is a simple test to to: replace e820 and e801 with invalid values. This is what euan did, and this was the first patch released. With these values replaced with e800 the loader reports only 64MB of ram, ie it uses the fallback method 3 (ah=0x88). This is a great step because we are now sure about what function to patch.
We have 2 informations to change: the memmap content, and the memory size. These are the 2 elements that are returned. In the previous section, we saw that the size is calculated at the end from the content of the memmap array. This means that if we patch the memmap array before, the size will be correct (ie also patched).
As usual, the first question that comes when doing reverse engineering is: Where will we put the new code. Since we are in raw asm, there are not a lot of solutions. Appending our code at the end may crash everything: If once the loader is mapped in memory, this area is used to store datas (variables), our code will be overflowed. To avoid any problems, we must remove a part of the code. This means that one of the fallback methods will not be included in the patched loader. I think that removing the fallback 2 is a good choice: When we fall here we already failed in 3 methods the last of which should be supported since 1994. Anyone has a 1GB pc of this date? It is best to keep the last method because if it is a default mapping, it will allow to boot on such machines. Another point is that it was the last chance, so let's keep it the last chance to boot!
Now the real work begins. Euan and me first tried with several mask based algos to limit the amount of total memory, but this can not work. To get something working on all configurations, the following algo is used. This is called for each memmap entry that is read:
if(current_memory_size >= MAX_MEM_SIZE)
goto scan_next_entry;
if( (p_Memmap->extend + p_Memmap->base) > MAX_MEM_SIZE)
{
i_ToMuchMemSize = p_Memmap->extent + p_Memmap->size;
i_ToMuchMemSize -= MAX_MEM_SIZE;
p_Memmap->extend -= i_ToMuchMemSize;
}
The first point is that if we already reached the upper limit, we do nothing of the current entry, and step throw the next entries. The next ones will obviously not be processed but it allows to end the hooked algo the same way than for normal cases (ie it ends when the bios says that there is no more entries). Next we check the current memmap entry to see if its size value is to big. If it is to big, we trunc it to the max allowed size. This gives the following asm code:
00000778 81FD00000010 cmp ebp,0x10000000 ; current mem size is above 256MB? 0000077E 0F83E3FEFFFF jnc near 0x667 ; scan next entry but do nothing of the current 00000784 8B54241C mov edx,[esp+0x1c] 00000788 8B02 mov eax,[edx] ; eax = curmap.base 0000078A 034208 add eax,[edx+0x8] ; eax += curmap.extent 0000078D 3D00000010 cmp eax,0x10000000 ; if eax < 256MB 00000792 0F8E85FEFFFF jng near 0x61d ; jump to process entry 00000798 8B4A08 mov ecx,[edx+0x8] ; ecx = curmap.extent 0000079B 2D00000010 sub eax,0x10000000 ; eax -= 256 MB 000007A0 29C1 sub ecx,eax ; ecx = ecx - eax 000007A2 894A08 mov [edx+0x8],ecx ; hook memmap.extent value 000007A5 E973FEFFFF jmp 0x61d ; jump to process entry
This piece of code is called from 0x6b9, ie each time a memmap entry is read. Moreover the out condition of fallback method1 is modified to jump to the fallback method3. With this patch, memory detection should be limited to 256MB on any memory configuration. Euan released the patch with a limitation of 512MB
There is one last think to note: When I tried this patch, forcing the 2 first methods to fail does not allow to boot. This means either that the fallback method 3 is not a default mapping one, either that there is a bug in the redirections from fallback 1 to fallback3. This also means that such a loader will not boot on a pc that does not support e820 and e801.
This was all suggested and done by mmu_man. The idea here is to hook the int 15 bios calls to fool the loader. He published the source code of it's TSR here. This has to be installed on a dos boot floppy that starts the beos loader.
Using such a hack to make beos boot is funny because TSRs are DOS programs. A tsr is a "Terminate and Stay Resident" program. When they are launched they do some initialisation (this is the transient part of the program), and then exit. The interesting point of TSRs is that when they exit, not all the memory they used is restored: one part (the resident part) stays allocated and will execute some code triggered by an event. This event is an interrupt. To handle this interrupt, the transient part of the code must register one of its functions in the interrupt table entry corresponding to the interrupt to hook. Some time ago it was a common way to do some "multitasking" on DOS. We will use it here to hook the interrupt that is called to get the memory size: int15.
More informations on TSR can be found in [AOA18]
The code is quite clear. The transient part hooks int15 to int15hook. When an int15 is triggered, the TSR is called. int15hook first checks that the call has ax=0xe820 (get system memory map). if it is the case, we jump to int15cont. Just after is the hack part:
; try to hack the ram map mov eax,[es:di+16] cmp eax,00000001h ; is it a RAM range ? jnz hack_out ; no mov eax,[es:di+8] cmp eax,MAXRAM ; is the range larger than we want ? jle hack_out ; no ; HACK baby !!! mov eax,MAXRAM ; force limit len mov [es:di+8],eax
That's all! This little program will be loaded by "config.sys" when the boot floppy will start. The drawback of this method is that you have to boot beos from dos, but it works. Note that this code is not finished: for example if we have 2 entries with a value lower than MAXRAM, but the sum of the 2 is bigger, we will not hack the second memory entry. Mmu_man released this code "as is" due to lack of time.
Here we are. This is all for this time. I hope that you learned interesting things here. This was a very interesting work because we tried to do the same thing at different places and learned a lot on each part. With this patch BeOS will be available to boot all new machines until openbeos come out!
Just one last thing: As you noticed (I hope), I did only a small part of what has been described above. Most of the work was done by Euan and mmu_man. There has been also a lot of contributions on the beosjournal forum.
See you later, Zadig.
[OSIDP] :
Operating Systems, Internals and Design Principles
Part 3, Memory
[BROWN] :
Ralph Brown's interrupt list
http://www.ctyme.com/intr/int.htm
[AOA18] :
The art of assembly
chapter 18, Resident Programs
http://webster.cs.ucr.edu/AoA/DOS/pdf/ch18.pdf
[PIRGE] :
Reverse Engineering the BeOS Boot Sequence Images, by Pirge
http://pirge.cjb.net/projects/zbeosimages.txt
#include#include /* lazy - for zbeos this is big enough */ #define SIZE 2048000 static int ID1 = 0x1f; static int ID2 = 0x8b; static int CM = 0x08; static int ID_BYTES = 3; int main (int argc, char* args[]) { FILE* fin; FILE* fout; char outfile[50]; int i; int numgzips = 0; long begin = 2147438647; long end = 0; int buf[SIZE]; int finished = 0, first_is_found = 0; printf("*******************************************\n"); printf("*\n"); printf("* dezbeos - extracts gzip files from zbeos\n"); printf("* - bru::pirge 07-12-2001\n"); printf("* - thanks to Amino"); printf("*\n"); printf("*******************************************\n"); fin = fopen("zbeos", "rb"); if(!fin) { printf("Could not read zbeos!\nPlease run in the same directory as zbeos\n"); exit(1); } /* read file and extract gzip files */ while(!feof(fin)){ /* is this a gzip file */ if( fgetc(fin) == ID1 && fgetc(fin) == ID2 && fgetc(fin) == CM) { /* set begin offset */ begin = ftell(fin); /* count gzip files found */ numgzips++; printf("Found gzip file %d at offsets: start 0x%x ", numgzips,begin-ID_BYTES); /* save gzip we have found */ sprintf(outfile,"found%d.gz",numgzips); /* find end of gzip - eof or another gzip */ while(!feof(fin)) { /* set end offset */ end = ftell(fin); /* is next a gzip id? */ if( fgetc(fin) == ID1 && fgetc(fin) == ID2 && fgetc(fin) == CM) { finished = 1; break; } } /* if its the end of file then set end offset */ if(finished == 0) end = ftell(fin); /* reset begin offset to include the two id bytes */ /* and the file pointer */ begin -= ID_BYTES; printf("end 0x%x\n",end); /* extract the boot part */ if(first_is_found == 0) { fseek(fin,0,0); /* open out file */ fout = fopen("boot.img","wb"); if(!fout) { printf("could not open %s\n","boot.img"); exit(1); } /* write out the rest */ fread(buf,sizeof(unsigned char),begin,fin); fwrite(buf,sizeof(unsigned char),begin,fout); /* close new file */ close(fout); printf("Extracted boot : %d bytes\n", begin); first_is_found = 1; } fseek(fin,begin,0); /* open out file */ fout = fopen(outfile,"wb"); if(!fout) { printf("could not open %s\n",outfile); exit(1); } /* write out the rest */ fread(buf,sizeof(unsigned char),end-begin,fin); fwrite(buf,sizeof(unsigned char),end-begin,fout); /* close new file */ close(fout); /* reset file pointer */ fseek(fin,end,0); } } /* close input */ close(fin); printf("Use: 'gzip -N -d found*.gz' to decompress the gzip files.\nFinished\n"); return 0; }
#!/bin/sh echo "**************************" echo "* loader_extract v1.0 *" echo "* by Zadig - 03/2004 *" echo "**************************" if [ "$1" == "" ] ; then echo "usage : loader_extract [loader_name]" exit fi if [ ! -f "$1" ] ; then echo "file $1 not found!" exit fi zbextract $1 gzip -N -d found1.gz gzip -N -d found2.gz
#!/bin/sh echo "**************************" echo "* loader_compress v1.0 *" echo "* by Zadig - 03/2004 *" echo "**************************" if [ "$1" == "" ] ; then echo "usage : loader_compress [base path] [loader_name]" exit fi if [ "$2" == "" ] ; then echo "usage : loader_compress [base path] [loader_name]" exit fi if [ ! -d "$1" ] ; then echo "directory $1 not found!" exit fi if [ ! -e "$1/boot.img" ] ; then echo "file $1/boot.img not found!" exit fi if [ ! -e "$1/images" ] ; then echo "file $1/images not found!" exit fi if [ ! -e "$1/beos" ] ; then echo "file $1/beos not found!" exit fi # compress loader files gzip -f -9 "$1/images" gzip -f -9 "$1/beos" #make loader cat "$1/boot.img" > $2 cat "$1/beos.gz" >> $2 cat "$1/images.gz" >> $2 #restore loader files gzip -N -d "$1/beos.gz" gzip -N -d "$1/images.gz"