Julia metaprogramming

May 14, 2021 Julia

Metaprogramming

Like Lisp, Julia's own code is also the data structure of the language itself. B ecause the code is represented by objects constructed and processed by the language itself, programs can also convert and generate code in their own language. Another function of metaprogramming is reflection, which dynamically presents the characteristics of the program itself while it is running.

Expressions and evaluations

The Julia code represents a syntax tree made up of Expr data structure. Here is the definition of the Expr type:

type Expr
  head::Symbol
  args::Array{Any,1}
  typ
end

head is a symbol that indicates the type of expression; args may be symbols that reference variable values at evaluation, Expr or they may be real object values. typ are inferred by types to make type comments and can often be ignored.

There are two ways to "reference" code, and they can simply construct expression objects without explicitly constructing Expr T he first is an inline expression, using : , followed by a single expression, and the second is a block of code placed in quote ... end internal. The following example is the first method that refers to an arithmetic expression:

julia> ex = :(a+b*c+1)
:(a + b * c + 1)

julia> typeof(ex)
Expr

julia> ex.head
:call

julia> typeof(ans)
Symbol

julia> ex.args
4-element Array{Any,1}:
  :+
  :a
  :(b * c)
 1

julia> typeof(ex.args[1])
Symbol

julia> typeof(ex.args[2])
Symbol

julia> typeof(ex.args[3])
Expr

julia> typeof(ex.args[4])
Int64

The following example is the second approach:

julia> quote
         x = 1
         y = 2
         x + y
       end
quote  # none, line 2:
    x = 1 # line 3:
    y = 2 # line 4:
    x + y
end

Symbol

: When the argument to is a symbol, the result Symbol object, not Expr

julia> :foo
:foo

julia> typeof(ans)
Symbol

In the context of an expression, a symbol is used to indicate a read of a variable. When an expression is evaluated, the value of the symbol is limited by the scope of the symbol (see scope of the variable).

Sometimes, to prevent ambiguity when parsing, : of : need to be added with additional parentheses:

julia> :(:)
:(:)

julia> :(::)
:(::)

Symbol can also be symbol using the symbol function, with an argument of one character or string:

julia> symbol('\'')
:'

julia> symbol("'")
:'

Value and interpolation

Specify an expression that Julia can evaluate in the global scope using the eval function.

julia> :(1 + 2)
:(1 + 2)

julia> eval(ans)
3

julia> ex = :(a + b)
:(a + b)

julia> eval(ex)
ERROR: a not defined

julia> a = 1; b = 2;

julia> eval(ex)
3

Each component has an eval expression that evaluates the evaluation expression eval Expressions eval are not limited to returning a value - they can also have side effects that change the environmental state of a closed module:

julia> ex = :(x = 1)
:(x = 1)

julia> x
ERROR: x not defined

julia> eval(ex)
1

julia> x
1

An expression is simply Expr object that can be programmed and then evaluated:

julia> a = 1;

julia> ex = Expr(:call, :+,a,:b)
:(+(1,b))

julia> a = 0; b = 2;

julia> eval(ex)
3

Note the difference between a b the example above:

When an expression is constructed, the value of the variable a a used directly. Therefore, there is no effect on a the expression is evaluated: the value in the expression is 1 independent of a value of a now
When an expression is constructed, the symbol :b T herefore, the value of the b the time of construction is irrelevant -- :b just a symbol, at which point the variable b has b been defined. When evaluating an expression, the value of the b is resolved by querying :b b

It's ugly to construct an Expr object like this. J ulia allows interpolation of expression objects. So the example above can be written as:

julia> a = 1;

julia> ex = :($a + b)
:(+(1,b))

The compiler automatically translates this syntax into the Expr on it.

Code generation

Julia uses expression interpolation and evaluation to generate duplicate code. The following example defines an operator for a set of operations with three parameters: :

for op = (:+, :*, :&, :|, :$)
  eval(quote
    ($op)(a,b,c) = ($op)(($op)(a,b),c)
  end)
end

Examples can be : reference format to write more streamlined: ::

for op = (:+, :*, :&, :|, :$)
  eval(:(($op)(a,b,c) = ($op)(($op)(a,b),c)))
end

It eval(quote(...)) mode for in-language code generation. Julia uses macros to short-write this pattern: ::

for op = (:+, :*, :&, :|, :$)
  @eval ($op)(a,b,c) = ($op)(($op)(a,b),c)
end

@eval call with a macro, making the code leaner. @eval the code can also be block code:

@eval begin
  # multiple lines
end

Interpolation of non-referenced expressions throws compile-time errors:

julia> $a + b
ERROR: unsupported or misplaced expression $

Macro

Macros are a bit like compile-time expression generators. J ust as a function gets a return value from a set of parameters, macros can transform expressions that allow programmers to convert expressions arbitrarily in the final program syntax tree. The syntax of calling a macro is:

@name expr1 expr2 ...
@name(expr1, expr2, ...)

Note that the macro name is signed @ sign. I n the first form, there are no commas between parameter expressions, and in the second, there are no spaces after the macro name. D on't confuse the two forms. F or example, the following method of writing results in a different way than the example above, which passes only one (expr1, expr2, ...)

@name (expr1, expr2, ...)

Before the program runs, @name the function to process the expression argument, replacing the expression with the result. Use the keyword macro to define an expanded function:

macro name(expr1, expr2, ...)
    ...
    return resulting_expr
end

The following example is a @assert in Julia:

macro assert(ex)
    return :($ex ? nothing : error("Assertion failed: ", $(string(ex))))
end

This macro can be used as follows:

julia> @assert 1==1.0

julia> @assert 1==0
ERROR: Assertion failed: 1 == 0
 in error at error.jl:22

The macro call is expanded to return the result at resolution. This is equivalent to:

1==1.0 ? nothing : error("Assertion failed: ", "1==1.0")
1==0 ? nothing : error("Assertion failed: ", "1==0")

The above code means that when the expression :(1==1.0) is stitched together as a conditional string(:(1==1.0)) with an assertion. S o all of these expressions make up the syntax tree of the program. T hen, during the run, if the expression is nothing and if the condition is false, a prompt statement will indicate that the expression is false. Note that there is no substitute for a function here, because only values can be passed in a function, and if we do, we cannot get a specific expression in the final error result.

The definition of real @assert standard library is more complex, allowing users to manipulate error messages, not just print them out. As with functions, macros can also have variable parameters, and we can look at this definition below:

macro assert(ex, msgs...)
    msg_body = isempty(msgs) ? ex : msgs[1]
    msg = string("assertion failed: ", msg_body)
    return :($ex ? nothing : error($msg))
end

Now, based on the number of receipts of @assert the data into two modes of operation. I f there is only one argument, the expression is captured empty by msgs and is described above as a simpler definition. I f the user fills in the second argument, the argument is used as a print argument instead of an incorrect expression. You can check the results of macro extensions in the following function called macroexpand

julia> macroexpand(:(@assert a==b))
:(if a == b
        nothing
    else
        Base.error("assertion failed: a == b")
    end)

julia> macroexpand(:(@assert a==b "a should equal b!"))
:(if a == b
        nothing
    else
        Base.error("assertion failed: a should equal b!")
    end)

In the @assert definition, there is another situation: what if we want to print not just "a should equal b," but also want to print their values? S ome people may naively want to insert string variables such @assert a==b "a ($a) should equal b ($b)!" this macro will not execute as we would like. C an you see why? Looking back at the chapter of the string, a string override function, compare:

julia> typeof(:("a should equal b"))
ASCIIString (constructor with 2 methods)

julia> typeof(:("a ($a) should equal b ($b)!"))
Expr

julia> dump(:("a ($a) should equal b ($b)!"))
Expr
  head: Symbol string
  args: Array(Any,(5,))
    1: ASCIIString "a ("
    2: Symbol a
    3: ASCIIString ") should equal b ("
    4: Symbol b
    5: ASCIIString ")!"
  typ: Any

So now you shouldn't get a string msg_body which receives the entire expression and needs to be evaluated as we expect. T his can be stitched directly into the returned expression as an argument to the string call. Get the full implementation by looking at the error.jl source code.

@assert greatly simplify expressions through macro replacement.

Sanitary macros

Health Macro is a more complex macro. I n general, macros must ensure that the introduction of variables does not conflict with existing context variables. C onversely, expressions in macros, as arguments, should be able to interact organically with context code. A nother concern is whether macros should be called another pattern when they are defined in different ways. I n this case, we need to make sure that all global variables should be incorporated into the correct pattern. J ulia already has a big advantage over other languages (such as C) in macros. All variables, @assert msg file, follow this criterion.

Take a look @time macro, whose argument is an expression. It records the time, runs the expression, then records the time, prints the time difference between the two times, and its final value is the value of the expression:

macro time(ex)
  return quote
    local t0 = time()
    local val = $ex
    local t1 = time()
    println("elapsed time: ", t1-t0, " seconds")
    val
  end
end

t0 t1 and val temporary variables, and time is a time time library, not a time that the user might use println function).

This is how the Julia macro unfold mechanism resolves naming conflicts. F irst, the variables of the macro result are classified as local or global variables. A variable is considered a local variable if it is assigned (and not declared a global variable), declared as a local variable, or used as a function argument name; The local variable is renamed to a unique name (a new symbol gensym and the global variable is parsed into the macro definition environment.

But there is still a problem that remains unsolved. C onsider the following example:

module MyModule
import Base.@time

time() = ... # compute something

@time time()
end

In this case, ex call to time but it is not the time function used time macro. I t actually points MyModule.time S o we should make changes to the ex code that we want to ex the macro calling environment. This is esc the expression to the esc function:

macro time(ex)
    ...
    local val = $(esc(ex))
    ...
end

In this way, encapsulated expressions are not handled by the macro-expanding mechanism and can be properly resolved in the macro call environment.

If necessary, this escape mechanism can be used to "destroy" hygiene by introducing or operating custom variables. The following example sets x to 0 x calling environment:

macro zerox()
  return esc(:(x = 0))
end

function foo()
  x = 1
  @zerox
  x  # is zero
end

This should be used with caution.

Non-standard string text

String text with identifier prefixes discussed in Strings is called non-standard string text and has special semantics. For example:

r"^\s*(?:#|$)" regular expression object instead of a string
b"DATA\xff\u2200" an array of bytes of text [68,65,84,65,255,226,136,128]

In fact, these behaviors are not built into the Julia interpreter or encoder, they call macros with special names. F or example, a regular expression macro is defined as follows:

macro r_str(p)
  Regex(p)
end

Therefore, r"^\s*(?:#|$)" equivalent to putting the following objects directly into the syntax tree:

Regex("^\\s*(?:#|\$)")

Writing this is not only short string text, but also efficient: regular expressions need to be Regex is constructed only when the code is compiled, so it is compiled only once, not every execution. In the following example, there is a regular expression in the loop:

for line = lines
  m = match(r"^\s*(?:#|$)", line)
  if m == nothing
    # non-comment
  else
    # comment
  end
end

If you don't want to use macros, to compile the above example only once, you need to override it as follows:

re = Regex("^\\s*(?:#|\$)")
for line = lines
  m = match(re, line)
  if m == nothing
    # non-comment
  else
    # comment
  end
end

For compiler optimization reasons, the above example is still not as efficient as using macros. Sometimes, however, it may be more convenient not to use macros: you must use this troublesome way to interpolate regular expressions;

More than standard string text, the command text syntax echo "Hello, $person" is also implemented with macros:

macro cmd(str)
  :(cmd_gen($shell_parse(str)))
end

Of course, a lot of complex work is hidden by the functions in this macro definition, but they are also written by Julia. Y ou can read the source code and see how it works. All it does is construct an expression object that is inserted into the syntax tree of your program.

Reflection

In addition to reflection at the level of metaprogramming syntax, Julia provides some other runtime reflection capabilities.

Type Field The name of the domain (or module member) of the data type can be names using the names command. For example, given the following type:

type Point
  x::FloatingPoint
  y
end

names(Point) will return the Any[:x, :y] The type of each domain in a Point is stored in the types field of types pointer object:

julia> typeof(Point)
DataType
julia> Point.types
(FloatingPoint,Any)

Sub-type

Direct subtypes of any data type subtypes(t::DataType) F or example, the abstract data FloatingPoint four (specific) subtypes:

julia> subtypes(FloatingPoint)
4-element Array{Any,1}:
 BigFloat
 Float16
 Float32
 Float64

Any abstract subsype will also be included in this list, but not further subsepes;

Inside the type

The internal representing of the type is important when using the C code interface. isbits(T::DataType) T is stored in C language compatible positioning. The amount of compensation within each domain can fieldoffsets(T::DataType) statements.

The function method

All methods within the function methods(f::Function) statements.

The function represents

Functions can implement internal checks at several levels of display. A lower form of a function code_lowered code_lowered(f::Function, (Args...)) i s available, while the lower form of type code_typed(f::Function, (Args...)) is available.

Closer to the machine, the function represented in the middle of the LLVM is printed by code_llvm(f::Function, (Args...)) and the resulting assembly code_native(f::Function, (Args...)