A good VM is a constantly changing VM

Photo by Robin De Kesel / Unsplash

Linear disassembly

Your disassembler goes through decoding every opcode in the bytecode linearly.

Here is an example of where linear disassembly works

VM Bytecode:

0, 4, 0, 6, 1

key generated or created by attacker:

0 = op_number, 1 subsequent num argument 1 = op_add

Step through bytes linearly to disassemble:

op_number, 4 op_number, 6 op_add

---

Simple, but If the VM changes things around during runtime, like opcode IDs or how instructions are decrypted, just going through instruction by instruction will not work since there are bytecode modifiers that change how the bytecode is interpreted at runtime.

Linearly decoding a VM that does this runtime modification would lead to an invalid disassembly because linearly moving through the bytecode would not preform the bytecode modification operations in the right order.

That is when you are forced to use

Recursive disassembly

Your disassembler follows jumps and if statements while decoding every opcode.

Here is an example of where recursive disassembly works

VM Bytecode:

0, 4, 9, 3, 5, 11, 12, 8, 3, 9, 4

With incrementation removed (for clarity)

0, 4, 9, 3, 2, 8, 9, 5, 0, 6, 1

Key generated or created by attacker:

0 = op_number, 1 subsequent num argument 1 = op_add 2 = op_jump, 1 subsequent num argument 9 = op_add_n_to_op_ids, 1 subsequent num argument

Step through bytes recursively to disassemble:

op_number, 4 op_add_n_to_op_ids, 3

(every opcode’s id from here on is incremented by 3)

op_jump, 8

(this skips over the second op_add_n_to_op_ids, if we counted that then all subsequent ops would be wrong, it is essentially dead code)

op_add_n_to_op_ids, 5 (not executed)

op_number, 6 op_add

---

"I’d prefer linear when it’s possible because you get everything there is, and recursive might skip something etc. But sometimes you have to use a recursive approach, because they either put random bytes after a jump so the disasm just breaks or they change their opcode order after a jump" - Kian, SneakerDev Discord

Google/Botguard Disassembly - CSolver, SneakerDev Discord

You can do linear disassembly for Kasada(No longer possible), but not for Google/Botguard since

"They bring in new opcodes during runtime which are reconstructed into strings then eval’d for usage, and modify both the encryption keys & the seed which have to match with the byte position. They use keys/seed to determine the next byte, so when changed, the control flow changes with it. That would be a prime example for when linear would not work." - CSolver, SneakerDev Discord

This changing of bytecode during runtime forces any reverse engineer attempting a disassembly to take a more active approach to it.

Slightly unrelated but here’s some other interesting information about Google/Botguard.

"The anti debug feature is easily one of the hardest parts. They hide it very well while having a couple anti debug functions easily visible. So you would think u removed all anti debug and patched it, but there will still be more. They also have integrity checks for modifications. and they override a lot of prototypes so logging variables changes things in the program. their timing checks are very precise too" - CSolver, SneakerDev Discord

"for example if they override .toString and you try to log a variable then it executes another function which modifies the variable" - CSolver, SneakerDev Discord