(untitled)

Do you know about any tools/algorithm used to identify raw binary data (especially raw binary code, like x64 or ARM instructions). I am pretty sure that it can be done with some kind of probability

>>2863 file, binwalk. file mostly just checks magic numbers then depending on what it is, digs into the metadata for that filetype. binwalk checks for magic numbers inside an existing binary file. If it finds an archive it can identify things inside it recursively with the -r flag.

>>2864 Yeah, but there is no magic numbers, binary code only (for example shellcode)

>>2865 You're asking if it's possible to determine what purpose an arbitrary set of bytes has? The short answer is "No."

>>2867 But it is not "arbitrary set of bytes" I know the problem is very hard, but some architectures have slight differences in machine code, so there should exist some kind of a heuristic solution

>>2868 From what I understand OP has an arbitrary set of bytes and wants to know if it is code or not. Even if you run a disassembler over it and it all decodes to valid instructions it could still be gibberish. I'm not sure how you could solve this problem algorithmiclly.

there is no major tool for this purpose, and the only way to be sure that the instructions you get out of some random data aren't pure gibberish. Generally ISA's use the most compact set of bits they can, so there are very few numbers that can be simply discounted as illegal instructions. And even if you did search the entire hunk for illegal instructions, they might just be miscellaneous pieces of data that are in the binary. Basically, there's no way to know without trying to run the code.

The primary issue is this: a machine code program that needs to survive such scrutiny will simply encrypt itself. It's entirely feasible to write a payload that appears to be, say, ASCII, with the first few bytes being a simple loop that will unencrypt the rest of the data when being executed. The encryption will be something simple such as NOTing or XORing the data. This is one reason many systems have an option that prevents code from modifying data and then executing it. So, to do this, not only would you need to know where it would be executed, you'd also need to check each possible loop, and this is assuming that the loop is about what I described and not intentionally trying to cause a fault or whatnot. In brief, there's no good way to do this asides from having an intelligent actor doing it manually.

>>2893 Op here I know about self modyfying code, I've even written my own simple xor encoding binary protector. But I assume only "normal programs", without any obfuscation techniques. Interesting approach could be to test for compression ratio or some other algorithm which takes data as input and outputs number from given range

There's some research on doing this for natural language. The same techniques should apply for machine language or any other data. See: https://en.wikipedia.org/wiki/Language_identification I don't know that anyone has done this yet. There's a thesis idea for you. >>2897 > compression ratio This is actually one of the techniques mentioned in the article.