I remember learning these properties in basic algebra: Associative, Distributive, and Commutative. It’s the Commutative property that states that a + b = b + a. The same principle is true with multiplication. In x86 pointer math, of course the results of these operations follow the commutative property; that’s just math. However, the machine encoding doesn’t consistently take this into account. To be facetious with the blog title image, machine code takes apple color into account most of the time, assembly language just looks at the number of apples.
To spoil everything from the beginning, ‘xor byte [esp + eax], 0’ is encoded the same as ‘xor byte [eax + esp], 0’. In machine code, when using esp as one of two non-scaled registers as a pointer, the commutative property is acknowledged. However, any other of the 8 general purpose registers are not treated this way! In other words, ‘xor byte [ebx + ecx], 0’ is not encoded the same as ‘xor byte [ecx + ebx], 0’
The claim I just made is slightly unfair though, or unfair-ish. Assemblers (like nasm) allow us to get kind of loose with our assembly. An instruction like ‘xor byte [ebx + ecx], 0’ isn’t really showing the whole story. In assembly, these pointers are made of up 3 parts (all 3 are optional…ish): one base register, one scaled register, and one displacement (8/32 bit offset). Scaled registers can be multiplied by 1, 2, 4, or 8. So more accurately, the above instruction is actually a base register of ‘ebx’ + a scaled register of ‘ecx’ with a scale of 1 (multiplied by 1). In machine code, the encoding of a base register and scaled register are encoded entirely differently.
With the above knowledge in mind, it’s no surprise that ‘xor byte [ebx + ecx * 1], 0’ is not the same as ‘xor byte [ecx + ebx * 1], 0’. Even if the result of what memory location this points to is the same (it is), it is now obvious why these are encoded differently…except for when it’s the ‘esp’ register…
When writing assembly, ‘xor byte [eax + esp * 1], 0’ would get encoded the same as ‘xor byte [esp + eax * 1], 0’. The actual encoding for both more accurately represents the 2nd form of this instruction: xor byte [esp + eax * 1]. Remember, the esp register can not be scaled Why ESP doesn’t scale (But EBP can still Base). If I were to write ‘xor byte [eax + esp * 2], 0’ instead of ‘* 1’, I would get an error from my assembler. Instead, my assembler (nasm) is clever enough to know that even though my instruction (scaling esp by 1) is not valid, it replaces it with an equivalent instruction (using the commutative property), and all is well. But without knowing machine code, this would all be happening magically behind the scenes to us, because assembly is too high level.
Before we forget, lets take a look at ebp, because the ebp register also gets encoded differently sometimes in memory pointers. Even though esp can not in any case be encoded as a scaled register, making ebp the base register can be done, but comes with a compromise: the displacement component of the pointer is no longer optional. So if you didn’t include a displacement in your assembly, the assembler will add a zeroed out byte as a displacement for you. In other words, ‘xor byte [ebp + eax], 0’ is actually more accurately ‘xor byte [ebp + eax + 0x00], 0’.
So taken the above information about the encoding of ebp, we are faced with a trade-off. This is one of those times where my assembler takes me literally, in the sense that it obeys making ebp the base (first) register (even though it may add an extra null byte behind our backs to do it). So even though, ‘xor byte [eax + ebp * 1], 0’ logically does the same thing (commutative property), nasm does not choose this form, because it doesn’t need to do this, like it does with the scaled esp register. The interesting thing is that this alternate form of the instruction is a byte less (because it doesn’t need that displacement byte). The takeaway is that if you are using two unscaled registers in your pointer, and ebp is one of them, and you didn’t already have a displacement: make ebp the last one (all to save one byte)