Some debugging tools for system developers

Here is some good assembly tools that worth recommendations.

* windows Debug.exe
The only 16 bit debugger that I know is debug.exe; it is free, and it comes with every modern windows system, just type 'debug a.exe' from command prompt to use it.

* dependency walker
A nice DLL dependency analyzer from microsoft. It will list different PE section information according to windows PE format.

* PE browser
A 3rd party free tool. Nice work! It is capable of de-assemble 32 bit executable, and comment it nicely.

* OllyDBG
Quite famous tool among assembly programmers. It is capable to de-assemble 32 bit executable, and trace code while watching memory and processor registers.

* Visual studio dumpbin
I have not tried yet. no comment.

please leave a comment if you know of any good de-assembler and executable analyzer. thanks for sharing!

some notes on masm

summary on c style calling convention:

1. the caller needs to push params to stack
2. params are pushed from right to left
3. caller needs to adjust the stack after 'call' (increase the sp)

bootup sequence from disk perspective -- a chinese version

http://211.90.241.130:22366/view.asp?file=309
罗云彬的网站教程

三. 系统启动过程简介 系统启动过程主要由一下几步组成(以硬盘启动为例):
1. 开机 :-)
2. BIOS 加电自检 ( Power On Self Test -- POST ) 内存地址为 0ffff:0000
3. 将硬盘第一个扇区 (0头0道1扇区, 也就是Boot Sector) 读入内存地址 0000:7c00 处.
4. 检查 (WORD) 0000:7dfe 是否等于 0xaa55, 若不等于 则转去尝试其他启动介质, 如果没有其他启动介质则显示 "No ROM BASIC" 然后死机.
5. 跳转到 0000:7c00 处执行 MBR 中的程序.
6. MBR 首先将自己复制到 0000:0600 处, 然后继续执行.
7. 在主分区表中搜索标志为活动的分区. 如果发现没有活动 分区或有不止一个活动分区, 则转停止.
8. 将活动分区的第一个扇区读入内存地址 0000:7c00 处.
9. 检查 (WORD) 0000:7dfe 是否等于 0xaa55, 若不等于则 显示 "Missing Operating System" 然后停止, 或尝试 软盘启动.
10. 跳转到 0000:7c00 处继续执行特定系统的启动程序.
11. 启动系统 ...

以上步骤中 2,3,4,5 步是由 BIOS 的引导程序完成. 6,7,8,9,10步由MBR中的引导程序完成.

the many difference between 16 bit asm and 32 bit asm

There are many things being different between 16 bit asm programming and 32 bit asm programming.

* The introduction of 32 bit segment registers. (EAX, EBX, etc)

* A program written in native 32 bit Windows format is created in what is called FLAT memory model which has a single segment that contains both code and data. The programs must be run on a 386 or higher processor.

* Differing from earlier 16 bit code that used combined segment and offset addressing with a 64k segment limit, FLAT memory model works only in offsets and has a range of 4 gigabytes. This makes assembler easier to write and the code is generally a lot faster.

* All segment registers are automatically set to the same value with this memory model and this means that segment / offset addressing must NOT be used in 32 bit programs that run in 32 bit Windows.

* For programmers who have written code in DOS, a 32 bit Windows PE executable file is similar in some respects to a dos COM file, they have a single segment that can contain both code and data and they both work directly in offsets, neither use Segment / Offset addressing. The defaults in flat-model programs are NEAR code addressing and NEAR data addressing within the range of 4 gigabytes.

* The addition of FS and GS segment registers in 32 bit mode. The FS and GS segment registers are not normally used in application programs but are used in some instances by the operating system.

* 32 bit mode has introduced the concept of priority which helps for memory organization and isolation.

some important but always ignored questions in ASM

These are some issues which bothered me a lot when I started to code ASM. I hope to note down and share it with people.

The first fact on ASM tutorial and sample code for beginners, most of them are written for 16 bit environment!!! And, what does that mean? it means you have to compile them with 16 bit assembler and link them with 16 bit linker. If you happen to start with Microsoft MASM v8 (like me), you may use the 'ml.exe' for 16 bit or 32 bit code assembling by a command option switch, however, the default 'Link.exe' included with the package is a 32 bit linker and it does not link 16 bit object files!!!

so??? so you have to download a separate 16 bit linker(supposeably, 'lnk16.exe'), and put it in the /bin directory of your assembler installation, and specifically use it.

now that is how your compile your 16-bit code,

C:\masm32\bin\ML /c /Cp /nologo /I"C:\masm32\include" A16bit.ASM
C:\masm32\bin\Lnk16 /SUBSYSTEM:CONSOLE A16bit.obj /o A16bit.exe

(the /c option force the ML.exe to do assembling only, no linking.
/Cp keep user case sensitive symbols)

or alternatively, you can compile and link in one go,
C:\masm32\ML /Bl .\Lnk16.exe /SUBSYSTEM:CONSOLE /Cp /nologo /I"C:\masm32\include" A16bit.ASM /o A16bit.exe

then how do we compile 32-bit code? using the /coff switch. This is how you compile them in one go,
C:\masm32\ML /coff /Cp /nologo /SUBSYSTEM:WINDOWS /I"C:\masm32\include" A16bit.ASM /o A16bit.exe

(this will use the default 'link.exe')

There are actually quite a lot of options for ML and link, and you can find them out by
ML /?
link /?

2007. June. 11
some more comment regarding the 16bit issue, when you see the following code snippet in asm, they mean to be compiled by 16 bit environment,
==================
mov ax, code ; or mov ax, @data
mov ds, ax
mov es, ax
==================
such statement is NOT allowed by 32bit compilers. Under 32 bit environment, segment concept is totally different.

G state, D state, C state, P state, what are they? -- ACPI

The following are extracted from wikipedia.org.

Global states

The ACPI specification defines the following seven states (so-called global states) an ACPI-compliant computer system can be in:

G0 Working is the normal working state of the computer, meaning that the operating system and whatever applications, run. The CPU(s) execute instructions. Within this state (i.e., without entering G1 Sleeping), it is possible for CPU(s) and devices like hard drives, DVD drives, etc. to be repeatedly put into and come back from low-energy states, called C0–Cn and D0–D3. (Laptops, for example, routinely power down all currently unused devices when running on battery; some computers do this to reduce noise.)
G1 Sleeping subdivides into the four states S1 through S4. The time needed to bring the system from here back into G0 Working (wake-latency time) is shortest for S1, short for S2 and S3, and not so short for S4.

S1 is the most power-hungry of sleep modes. All processor caches are flushed, and the CPU(s) stop executing instructions. Power to the CPU(s) and RAM is maintained; devices that do not indicate they must remain on may be powered down. Some newer machines do not support S1; older machines are more likely to support S1 than S3.

S2 is a deeper sleep state than S1, where the CPU is powered off; however, it is not commonly implemented.

S3 is called Standby in Windows, Sleep in Mac OS X, and sometimes also Suspend to RAM (STR), although the ACPI specification mentions only the terms S3 and Sleep. In this state, main memory (RAM) is still powered, although it is almost the only component that is. Since the state of the operating system and all applications, open documents, etc. lies all in main memory, the user can resume work exactly where he/she left off—the main memory content when the computer comes back from S3 is the same as when it was put into S3. (The specification mentions that S3 is rather similar to S2, only that some more components are powered down in S3.) S3 has two advantages over S4; the computer resumes in about the time it takes the monitor to come on, secondly if any running applications (open documents, etc) have private information in them, this will not be written to the disk. Desktop users using this state may want to disable disk caching (also called disk buffer) so that, in the event of a power failure, the system disk is less likely to become corrupted due to data in the cache not being written to the disk.

S4 -- Main article: Hibernate (OS feature). S4 is called Hibernation in Windows, Safe Sleep in Mac OS X, and sometimes also Suspend to disk, although the ACPI specification mentions only the term S4. In this state, all content of main memory is saved to a hard drive, preserving the state of the operating system, all applications, open documents etc. That means that after coming back from S4, the user can resume work where it was left off in much the same way as with S3. The difference between S4 and S3, apart from the added time of moving the main memory content to disk and back, is that a power loss of a computer in S3 makes it lose all data in main memory, including all unsaved documents, while a computer in S4 is unaffected. S4 is quite different from the other S states and actually resembles G2 Soft Off and G3 Mechanical Off more than it resembles S1–S3.
G2 (S5) Soft Off-- G2, S5, and Soft Off are synonyms. G2 is almost the same as G3 Mechanical Off, but some components remain powered so the computer can "wake" from input from the keyboard, LAN, or USB device [2]. This state is the same as G3 Mechanical Off, in that the boot procedure must be run to bring the system from G2 to G0 Working. G3 Mechanical Off is entered only when a power loss occurs, whereas G2 is initiated by the operating system (typically because the user issued a shutdown command in some way). The computer is not safe for disassembly in the G2 state due to the components that remain powered, however it is safe to unplug the computer and, after approximately 20 seconds, the computer will be in the G3 state. It is always advisable to unplug a desktop computer (and wait 20 seconds) prior to "disassembly", because, although internal drives are usually not powered, the (PS2) keyboard, USB ports, mainboard, expansion cards, and power supply may remain powered, even if the computer is not used to wake from input to these devices.
G3 Mechanical Off: The computer's power consumption is very close to zero, to the point that the power cord can be removed and the system is safe for disassembly (typically, only the real-time clock is running off its own small battery). The computer falls into this state when a power loss occurs, e.g. in case of a power outage. Once power is restored, a full boot procedure is necessary to bring the system from G3 to G0 Working.
Furthermore, a state Legacy is defined as the state when an operating system runs which does not support ACPI. In this state, the hardware and power are not managed via ACPI, effectively disabling ACPI.
(Reference: ACPI specification 3.0b is linked to under External links, section 7.3.4)

Device states

The device states D0-D3 are device-dependent:
D0 Fully-On is the operating state.
D1 and D2 are intermediate power states whose definition varies by device.
D3 Off has the device powered off and unresponsive to its bus.

Processor states

The CPU power states C0-C3 are defined as follows:
C0 is the operating state.
C1 (often known as Halt) is a state where the processor is not executing instructions, but can return to an executing state essentially instantaneously. Some processors, such as the Pentium 4, also support an Enhanced C1 state (C1E) for lower power consumption.
C2 (often known as Stop-Clock) is a state where the processor maintains all software-visible state, but may take longer to wake up.
C3 (often known as Sleep) is a state where the processor does not need to keep its cache coherent, but maintains other state. Some processors have variations on the C3 state (Deep Sleep, Deeper Sleep, etc.) that differ in how long it takes to wake the processor.

Performance states

While a device or processor is operating (D0 and C0, respectively), it can be in one of several power-performance states. These states are implementation-dependent, but P0 is always the highest-performance state, with P1 to Pn being successively lower-performance states, up to an implementation-specific limit of n no greater than 16.
P-states are called SpeedStep in Intel processors, Cool'n'Quiet in AMD processors and PowerSaver in VIA processors.

Key Benefits of the I/O APIC

The following contains extracts from MSDN and wikipedia.com.

The Intel APIC Architecture is a system of Advanced Programmable Interrupt Controllers (APICs) designed by Intel for use in Symmetric Multi-Processor (SMP) computer systems. It was originally implemented by the Intel 82093AA and 82489DX, and is found in most x86 SMP motherboards. It is one of several attempts to solve interrupt routing efficiency issues in multiprocessor computer systems.

There are two components in the Intel APIC system, the Local APIC (LAPIC) and the I/O APIC. The LAPIC is integrated into each CPU in the system, and the I/O APIC is used throughout the system's peripheral buses. There is typically one I/O APIC for each peripheral bus in the system. In original system designs, LAPICs and I/O APICs were connected by a dedicated APIC bus. Newer systems use the system bus for communication between all APIC components.

In systems containing an 8259 PIC, the 8259 may be connected to the LAPIC in the system's bootstrap processor (BSP), or to one of the system's I/O APICs.

Sharing Interrupts Is Bad

Edge-triggered Interrupts. With any interrupt controller, an edge-triggered interrupt is a one-time event. There is no notion of feedback. The operating system never really knows when it has handled the situation that caused the interrupt; it can only know that an event happened sometime in the recent past. Therefore, the only rational response to an edge-triggered interrupt is to run all the Interrupt Service Routines (ISRs) associated with that vector once, with the hope that this will resolve it.

This situation is especially sketchy when dealing with hardware that doesn't give any real indication of why it interrupted, a common occurrence among today's edge-triggering devices. The result is that the operating system can miss interrupts delivered in the interval between when an interrupt is first taken and when it is acknowledged.

With an 8259 PIC interrupt controller, the situation is even worse. The 8259 is inherently unreliable, particularly when coupled with an actual ISA bus. The operating system software will see a number of spurious interrupts, some of which show up on different vectors than the original signal.

Level-triggered Interrupts. The protocol of a level-triggered interrupt is as follows:
•Whenever the interrupt signal is held low (for active-low interrupts, which are the most common), the interrupt controller will generate an interrupt.
•If the interrupt is acknowledged and the signal is still low, then the interrupt controller will generate another interrupt.
This is good for sharing, because it confirms that the operating system handled the interrupting device. The algorithm is:
1.Take an interrupt.
2.Run the first ISR in the chain.
3.If that ISR returns TRUE (meaning that it handled an interrupt), then the operating system will ACK the interrupt and quit.
4.If that ISR returns FALSE, then run the next ISR in the chain and go to Step 3.

An excellent example of the problem case comes when there are twelve devices all chained on one vector, common for a docked laptop. Every time the operating system takes an interrupt, the operating system must run as many as twelve ISRs before it begins to handle the condition that caused the interrupt.

more on this question, http://www.microsoft.com/whdc/system/sysperf/IO-APIC.mspx

something on Assembly language (for beginners)

The different procedure calling convention.
STDCALL
Parameters are pushed to stack using right to left order.
Parameters pushed to stack are removed by the called procedure itself.

C ALL
Parameters are pushed to stack using right to left order.
Parameters pushed to stack are removed by the calling program.

PASCAL CALL
Parameters are pushed to stack using left to right order.
Parameters pushed to stack are removed by the called procedure itself.


Declaring Procedure Prototypes
MASM provides the INVOKE directive to handle many of the details important to procedure calls, such as pushing parameters according to the correct calling conventions. To use INVOKE, the procedure called must have been declared previously with a PROC statement, an EXTERNDEF (or EXTERN) statement, or a TYPEDEF. You can also place a prototype defined with PROTO before the INVOKE if the procedure type does not appear before the INVOKE. Procedure prototypes defined with PROTO inform the assembler of types and numbers of arguments so the assembler can check for errors and provide automatic conversions when INVOKE calls the procedure.

does it ring a bell on C prototype and forward declaration? :)

How to make a choice between a Macro and procedure?

Macros are processed by preprocessor, rather than the compiler. Macro invocations are replaced with the macro repetitively. It appears in the listing repetitively.

Procedures involves CALL and RET, and it appears only once in the listing;

So as a result, Macros has a larger size of compiled code, while Procedures has some overhead with CALL and RET, and thus a slower execution speed.

DDR-200, PC-1600, what is all this about

The following information is based on pages from wikipedia.com.

DDR SDRAM or double-data-rate synchronous dynamic random access memory is a class of memory integrated circuit used in computers. It achieves greater bandwidth than the preceding single-data-rate SDRAM by transferring data on the rising and falling edges of the clock signal (double pumped). Effectively, it nearly doubles the transfer rate without increasing the frequency of the front side bus. Thus a 100 MHz DDR system has an effective clock rate of 200 MHz when compared to equivalent SDR SDRAM, the “SDR” being a retrospective designation.

DDR-200: DDR-SDRAM memory chips specified to operate at 100 MHz
DDR-266: DDR-SDRAM memory chips specified to operate at 133 MHz
DDR-333: DDR-SDRAM memory chips specified to operate at 166 MHz
DDR-400: DDR-SDRAM memory chips specified to operate at 200 MHz

Then what is the PC-1600? it is the data rate,

Data rate = DDR frequency * 64(mem width) / 8 (bits per byte)

See the following table.

The Myth around A20 and HMA(High Memory Area)

The following is extracted from:
http://www.pcguide.com/ref/ram/logicHMA-c.html

this is the best explanation for this question I found so far.

PC memory addresses are referred to using a segment:offset addressing scheme; the segment is multiplied by 16 (shifted one hexadecimal digit to the left) and then the offset is added to it to get the real address. The original IBM PC had only 20 address lines and so its highest memory address was FFFFFh. However, the weird segment:offset addressing makes it possible to generate a linear address that is higher than this number.

Take the address FFFF:FFFF. To convert this to a linear address you take the segment and multiply it by 16, to get FFFF0. Then you add the offset, FFFF. FFFF0+FFFF in hexadecimal results in 10FFEFh. There's a problem with this: that "1" at the front requires a 21st address line to represent it, and that doesn't exist on the 8088 or 8086 processors used in the first PCs. They deal with this problem by ignoring the "1"--they treat the address simply as 0FFEFh. Software must be able to handle this "wrap around" of the memory addresses.

When Intel created the 80286 processor, it supported both protected mode and real mode. When in real mode the 80286 was supposed to behave exactly the same as an 8088 or 8086, for compatibility. However, the 80286 does have a 21st address line (it has 24), and due to a bug in its design it didn't do the wrap around in the same way as the 8088 when in real mode. When it used address FFFF:FFFF and came up with 10FFEFh for a linear address, it kept it as 10FFEFh instead of wrapping it around to 0FFEFh like on the older CPUs. This allowed the first FFEFh of extended memory (100000-10FFEFh) to be accessed by the chip even while still in real mode. This block of memory is the high memory area (HMA).

There was still the problem of ensuring compatibility of the 80286 when in real mode. IBM solved this in the original AT by using spare lines in the keyboard controller chip to manage the 21st address line (which is called the A20 line because address bits are numbered starting with zero). The keyboard controller turns off the A20 line when the processor is running in real mode, to allow for full compatibility with the older PCs. It turns it back on when running in protected mode.

Many years later, when the 640 KB limit of conventional memory began to be quite cramping, the ability to access an additional 64 KB of memory in real mode was seen as a significant advantage. (People at this time were scratching and clawing to get even 8 KB more conventional memory to let them run large programs that insisted on certain minimums). Microsoft developed a special driver called HIMEM.SYS that allowed the A20 line to be manipulated under software control. This allows the high memory area to be put to good use.
In practical terms, the high memory area is normally used by DOS itself. Specifying "DOS=HIGH" in the DOS system file CONFIG.SYS tells DOS to load a portion of its own code into the high memory area instead of into conventional memory. This frees approximately 45 KB of conventional memory for use by programs.

The final step in institutionalizing this former bug as an official PC feature was removing manipulation of the A20 line from the keyboard controller. Since that was originally a hack anyway--the controller was used because there was no better way to do it, after all, it has nothing to do with the keyboard--in many newer PCs there is a BIOS option to allow the chipset to control the A20 line directly. This provides a small performance increase compared to letting the keyboard controller manage the line.