Python Obfuscation Ast

2022-02-17

字数统计: 2.9k字 | 阅读时长≈ 15分

Python Ast用于混淆。

Python Obfuscation Ast

简介

和一般高级语言差不多，Python也有构建抽象语法树的一环。此ast模块就是用于此的。平时编程用处不多，但是由于能够操作ast结点，所以能实现混淆。

基本使用

解析

test = r"print('Hello')"

import ast
mod = ast.parse(test)
# <_ast.Module object at 0x0000023820536248>

打印

Py39以上支持indent参数，用于格式化打印。

ast.dump(mod)
# "Module(body=[Expr(value=Call(func=Name(id='print', ctx=Load()), args=[Str(s='Hello')], keywords=[]))])"

# ast.dump(mod, indent=4) 更美观

编译且执行

compile是内置函数，非ast模块内函数。

# compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)
"""
The source code may represent a Python module, statement or expression.
The filename will be used for run-time error messages.
The mode must be 'exec' to compile a module, 'single' to compile a
single (interactive) statement, or 'eval' to compile an expression.
"""
code = compile(mod, "<stdin", "exec")
# <code object <module> at 0x00000238205339C0, file "<stdin>", line 1>

"""
>>> exec(code)
Hello
"""

概念解释

Python代码的三种形式

高级语言源码

普通的.py文件。

AST抽象语法树

中间表示形式。一般用不上。供编译器使用，用于编译优化。本篇重点。

字节码

.pyc文件

完全转化成了字节码文件。供CPython VM虚拟机解释执行。人类难以阅读。

Modes

源代码到AST的编译过程可以以三种不同模式进行。AST的根节点依赖于传递给ast.parse()的mode参数，并且在从AST到目标代码的编译过程中传递给compile()传递相同的mode参数。

exec：默认值是mode='exec'。AST的根节点是ast.Module，根节点的属性body是一个子节点的列表。
eval：单独的表达式可以被编译为mode='eval'的AST对象，AST的根是一个ast.Expression，他的属性body是一个单节点，例如ast.Call或者ast.BinOp。把这个AST对象传递给eval()将会返回表达式的值。
single：单独的语句或者表达式可以被编译为mode='single'的AST对象，AST的根是ast.Interactive，它的属性body是一个子节点的列表。如果这是一个表达式，sys.displayhook()将会被调用并传入表达式的结果，就如同Python的交互式界面被调用了一样。

修正行列号

后文“删除节点”一块会用上。

编译出的AST对象，每一个节点都必须有行列号属性lineno并且col_offset。从普通源代码编译而来的AST对象已经具有了行列号。但是在程序中动态创建的节点没有行列好，有几个helper functions可以做修正行列号。

ast.fix_missing_locations() 通过父节点的行列号递归的修正所有缺失行列号的节点。
ast.copy_location()用于从另一个节点拷贝行列号. 特别是当做节点替换的时候有用。
ast.increment_lineno()增加一个节点和其子节点的行号，一般用于把代码块移动到其他位置。

to Source

Python标准库不提供一种从字节码到AST，或者从AST到源码转化的办法。但一些第三方工具可以实现这些：

astor可以把AST转化为可读的Python源码。
Meta also tries to decompile Python bytecode to an AST, but it appears to be unmaintained.
uncompyle6反编译器（反编译器的局限性都很大）

浏览与修改AST

NodeVisitor
NodeTransformer

`NodeVisitor`

class NodeVisitor(builtins.object)
 |  A node visitor base class that walks the abstract syntax tree and calls a
 |  visitor function for every node found.  This function may return a value
 |  which is forwarded by the `visit` method.
 |
 |  This class is meant to be subclassed, with the subclass adding visitor
 |  methods.
 |
 |  Per default the visitor functions for the nodes are ``'visit_'`` +
 |  class name of the node.  So a `TryFinally` node visit function would
 |  be `visit_TryFinally`.  This behavior can be changed by overriding
 |  the `visit` method.  If no visitor function exists for a node
 |  (return value `None`) the `generic_visit` visitor is used instead.
 |
 |  Don't use the `NodeVisitor` if you want to apply changes to nodes during
 |  traversing.  For this a special visitor exists (`NodeTransformer`) that
 |  allows modifications.
 |
 |  Methods defined here:
 |
 |  generic_visit(self, node)
 |      Called if no explicit visitor function exists for a node.
 |
 |  visit(self, node)
 |      Visit a node.

NodeVisitor用于浏览AST结点。官方文档上说不要使用此类来修改结点。但是我看很多博客照样用这个。不过官方特制的NodeTransformer在修改节点上显然更优秀一点。

实际上也是可以用于修改的。

修改操作符

m = "print(3%4)"
n = ast.parse(m)
v = NodeVisitor()

ast.dump(n)
"Module(body=[Expr(value=Call(func=Name(id='print', ctx=Load()), args=[BinOp(left=Num(n=3), op=Mod(), right=Num(n=4))], keywords=[]))])"

v.visit(n) # 遍历后将修改 n的AST树

ast.dump(n)
"Module(body=[Expr(value=Call(func=Name(id='print', ctx=Load()), args=[BinOp(left=Num(n=3), op=Add(), right=Num(n=4))], keywords=[]))])"
# 发现 Mod()被修改成了Add()

`NodeTransformer`

class NodeTransformer(NodeVisitor)
 |  A :class:`NodeVisitor` subclass that walks the abstract syntax tree and
 |  allows modification of nodes.
 |
 |  The `NodeTransformer` will walk the AST and use the return value of the
 |  visitor methods to replace or remove the old node.  If the return value of
 |  the visitor method is ``None``, the node will be removed from its location,
 |  otherwise it is replaced with the return value.  The return value may be the
 |  original node in which case no replacement takes place.
 |
 |  Here is an example transformer that rewrites all occurrences of name lookups
 |  (``foo``) to ``data['foo']``::
 |
 |     class RewriteName(NodeTransformer):
 |
 |         def visit_Name(self, node):
 |             return Subscript(
 |                 value=Name(id='data', ctx=Load()),
 |                 slice=Index(value=Str(s=node.id)),
 |                 ctx=node.ctx
 |             )
 |
 |  Keep in mind that if the node you're operating on has child nodes you must
 |  either transform the child nodes yourself or call the :meth:`generic_visit`
 |  method for the node first.
 |
 |  For nodes that were part of a collection of statements (that applies to all
 |  statement nodes), the visitor may also return a list of nodes rather than
 |  just a single node.
 |
 |  Usually you use the transformer like this::
 |
 |     node = YourTransformer().visit(node)
 |
 |  Method resolution order:
 |      NodeTransformer
 |      NodeVisitor
 |      builtins.object
 |
 |  Methods defined here:
 |
 |  generic_visit(self, node)
 |      Called if no explicit visitor function exists for a node.
 |
 |  ----------------------------------------------------------------------
 |  Methods inherited from NodeVisitor:
 |
 |  visit(self, node)
 |      Visit a node.
 |

删除结点

尝试删除if-else中的else分支

data = """
def nums():
    for i in range(2):
        if i % 2 == 0:
            print("even: ", i)
        else:
            print(" odd: ", i)

nums()
"""

"""
>>> exec(data)
even:  0
 odd:  1
"""

f_ast = ast.parse(data)

class NodeTransformer(ast.NodeTransformer):
    def visit_If(self, node):
        return ast.If(
            test=node.test,
            body=node.body,
            orelse=[]
        )
    
transformer = NodeTransformer()
f_ast = transformer.visit(f_ast) # 返回新的 AST
ast.fix_missing_locations(f_ast) # 填补缺失的lineo和col_offset属性，否则compile会报错

"""
>>> exec(compile(f_ast, 'a2', 'exec'))
even:  0
"""
# 发现else分支没了。

混淆

字符串混淆

将所有的字符串转变成两个子字符串相加的形式。

from ast import NodeTransformer, BinOp, Str, Add
 
class StringObfuscator(NodeTransformer):
    def visit_Str(self, node):
        return BinOp(left=Str(s=node.s[:len(node.s)//2]),
                     op=Add(),
                     right=Str(s=node.s[len(node.s)//2:])
                    )
    
m = "print('hello world')"
exec(m)
"""
>>> exec(m)
hello world
"""
n = ast.parse(m)
ast.dump(n)

s = StringObfuscator()

obfus_n = s.visit(n)	# 其实n和obfus_n一样，都被修改了
ast.dump(obfus_n)
ast.fix_missing_locations(obfus_n)
obfus_c = compile(obfus_n, 'obfused', 'exec')
exec(obfus_c)
# 没有区别，但是看AST树发现已经修改了。
# 当然如此弱的混淆，以至于在编译成字节码的时候都被优化掉了。

混淆示例

import ast
from ast import Assign, Name, Call, Store, Load, Str, Num, List, Add, BinOp
from ast import Subscript, Slice, Attribute, GeneratorExp, comprehension
from ast import Compare, Mult
import codegen
import random
import sys


def random_string(minlength, maxlength):
    return ''.join(chr(random.randint(0x61, 0x7a))
                   for x in range(random.randint(minlength, maxlength)))


def import_node(name, newname):
    """Import module obfuscation"""
    # import sys -> sys = __import__('sys', globals(), locals(), [], 0)
    return Assign(
        targets=[Name(id=newname, ctx=Store())],
        value=Call(func=Name(id='__import__', ctx=Load()),
                   args=[Str(s=name),
                         Call(func=Name(id='globals', ctx=Load()), args=[],
                              keywords=[], kwargs=None),
                         Call(func=Name(id='locals', ctx=Load()), args=[],
                              keywords=[], kwargs=None),
                         List(elts=[], ctx=Load()), Num(n=-1)],
                   keywords=[], kwargs=None))


def obfuscate_string(s):
    """Various String Obfuscation routines."""
    randstr = random_string(3, 10)

    table0 = [
        # '' -> ''
        lambda: Str(s=''),
    ]

    table1 = [
        # 'a' -> 'a'
        lambda x: Str(s=chr(x)),
        # 'a' -> chr(0x61)
        lambda x: Call(func=Name(id='chr', ctx=Load()), args=[Num(n=x)],
                       keywords=[], kwargs=None),
    ]

    table = [
        # 'abc' -> 'abc'
        lambda x: Str(s=x),
        # 'abc' -> 'a' + 'bc'
        lambda x: BinOp(left=Str(s=x[:len(x)//2]),
                        op=Add(),
                        right=Str(s=x[len(x)//2:])),
        # 'abc' -> 'cba'[::-1]
        lambda x: Subscript(value=Str(s=x[::-1]),
                            slice=Slice(lower=None, upper=None,
                                        step=Num(n=-1)),
                            ctx=Load()),
        # 'abc' -> ''.join(_x for _x in reversed('cba'))
        lambda x: Call(
            func=Attribute(value=Str(s=''), attr='join', ctx=Load()), args=[
                GeneratorExp(elt=Name(id=randstr, ctx=Load()), generators=[
                    comprehension(target=Name(id=randstr, ctx=Store()),
                                  iter=Call(func=Name(id='reversed',
                                                      ctx=Load()),
                                            args=[Str(s=x[::-1])],
                                            keywords=[],
                                            kwargs=None),
                                  ifs=[],
                                  is_async=False)
                    ])],
            keywords=[], kwargs=None),
    ]

    if not len(s):
        return random.choice(table0)()

    if len(s) == 1:
        return random.choice(table1)(ord(s))

    return random.choice(table)(s)


class Obfuscator(ast.NodeTransformer):
    def __init__(self):
        ast.NodeTransformer.__init__(self)

        # imported modules
        self.imports = {}

        # global values (can be renamed)
        self.globs = {}

        # local values
        self.locs = {}

        # inside a function
        self.indef = False

    def obfuscate_global(self, name):
        newname = random_string(3, 10)
        self.globs[name] = newname
        return newname

    def obfuscate_local(self, name):
        newname = random_string(3, 10)
        self.locs[name] = newname
        return newname

    def visit_Import(self, node):
        newname = self.obfuscate_global(node.names[0].name)
        self.imports[node.names[0].name] = newname

    def visit_If(self, node):
        if isinstance(node.test, Compare) and \
                isinstance(node.test.left, Name) and \
                node.test.left.id == '__name__':
            for x, y in self.imports.items():
                node.body.insert(0, import_node(x, y))
        node.test = self.visit(node.test)
        node.body = [self.visit(x) for x in node.body]
        node.orelse = [self.visit(x) for x in node.orelse]
        return node

    def visit_Str(self, node):
        return obfuscate_string(node.s)

    def visit_Num(self, node):
        d = random.randint(1, 256)
        # n/d * d + (n % d)
        return BinOp(left=BinOp(left=Num(node.n // d), op=Mult(), right=Num(n=d)),
                     op=Add(), right=Num(node.n % d))

    def visit_Attribute(self, node):
        if isinstance(node.value, Name) and isinstance(node.value.ctx, Load):
            node.value = self.visit(node.value)
            return Call(func=Name(id='getattr', ctx=Load()), args=[
                Name(id=node.value.id, ctx=Load()), Str(s=node.attr)],
                keywords=[], kwargs=None)
        node.value = self.visit(node.value)
        return node

    def visit_FunctionDef(self, node):
        self.indef = True
        self.locs = {}
        node.name = self.obfuscate_global(node.name)
        node.body = [self.visit(x) for x in node.body]
        self.indef = False
        return node

    def visit_Name(self, node):
        # obfuscate known globals
        if not self.indef and isinstance(node.ctx, Store) and \
                node.id in ('teamname', 'flag'):
            node.id = self.obfuscate_global(node.id)
        #elif self.indef:
            #if isinstance(node.ctx, Store):
                #node.id = self.obfuscate_local(node.id)
            #node.id = self.locs.get(node.id, node.id)
        node.id = self.globs.get(node.id, node.id)
        return node

    def visit_Module(self, node):
        node.body = [y for y in (self.visit(x) for x in node.body) if y]
        node.body = [y for y in (self.visit(x) for x in node.body) if y]
        return node


class GlobalsEnforcer(ast.NodeTransformer):
    def __init__(self, globs):
        ast.NodeTransformer.__init__(self)
        self.globs = {}

    def visit_Name(self, node):
        node.id = self.globs.get(node.id, node.id)
        return node

if __name__ == '__main__':
    if len(sys.argv) < 2:
        print ('Usage: python %s <pyfile>' % sys.argv[0])
        exit(0)

    if sys.argv[1] == '-':
        root = ast.parse(sys.stdin.read())
    else:
        root = ast.parse(open(sys.argv[1], 'rb').read())

    # obfuscate the AST
    obf = Obfuscator()
    root = obf.visit(root)

    # resolve all global names
    root = GlobalsEnforcer(obf.globs).visit(root)

    ast.fix_missing_locations(root)
    code = compile(root, 'a2', 'exec')
    exec(code)
    
    print(ast.dump(root))
    
    import dis
    dis.dis(code)
    # print(codegen.to_source(root))

取自于

Python Source Obfuscation using ASTs | Development & Security (jbremer.org)

且进行了修改，将其从Py27移植到Py3。

不过codegen模块似乎还是Py27的，所以有问题，暂时没用。

可视化（未实验）

1	pip install graphviz

def visit(node, nodes, pindex, g):
    name = str(type(node).__name__)
    index = len(nodes)
    nodes.append(index)
    g.node(str(index), name)
    if index != pindex:
        g.edge(str(index), str(pindex))
    for n in ast.iter_child_nodes(node):
        visit(n, nodes, index, g)
    
graph = Digraph(format="png")
tree = ast.parse(data)
visit(tree, [], 0, graph)
graph.render("test")