May 14, 2021 Julia
Julia uses inverted quotes to run external programs:
julia> `echo hello`
`echo hello`
It has several features:
Cmd
object to represent the command.
You can use this object to connect commands, run them, and read and write them through a pipeline
libc
and the output of the command points to
system
stdout
fork
exec
as
julia
processes of julia.
Here's an example of running an external program:
julia> run(`echo hello`)
hello
hello
the
echo
the echo command, which is sent to the standard output.
run
method itself
nothing
If the external command does not run correctly, an
ErrorException
thrown.
Use
readall
read the output of the command:
julia> a=readall(`echo hello`)
"hello\n"
julia> (chomp(a)) == "hello"
true
More generally, you can
open
to read from or write to an external command.
For example:
julia> open(`less`, "w", STDOUT) do io
for i = 1:1000
println(io, i)
end
end
Assign the file name to the variable
file
as an argument to the command.
Use $ for interpolation as in
$
text (see :ref:
man-strings
julia> file = "/etc/passwd"
"/etc/passwd"
julia> `sort $file`
`sort /etc/passwd`
If the file name has special characters, such
/Volumes/External HD/data.csv
it will look like this:
julia> file = "/Volumes/External HD/data.csv"
"/Volumes/External HD/data.csv"
julia> `sort $file`
`sort '/Volumes/External HD/data.csv'`
The file name is caused by single quotes. J
ulia knows
file
is interpolated as a single variable, and it automatically draws the content up. I
n fact, this is
file
value of file is not interpreted by the shell, so it does not need to be actually caused;
The following example can also work:
julia> path = "/Volumes/External HD"
"/Volumes/External HD"
julia> name = "data"
"data"
julia> ext = "csv"
"csv"
julia> `sort $path/$name.$ext`
`sort '/Volumes/External HD/data.csv'`
If you want to interpolate more than one word, you should use an array (or other iterative container):
julia> files = ["/etc/passwd","/Volumes/External HD/data.csv"]
2-element ASCIIString Array:
"/etc/passwd"
"/Volumes/External HD/data.csv"
julia> `grep foo $files`
`grep foo /etc/passwd '/Volumes/External HD/data.csv'`
If the array is interpolated as part of the shell word, Julia mimics the behavior
{a,b,c}
parameters:
julia> names = ["foo","bar","baz"]
3-element ASCIIString Array:
"foo"
"bar"
"baz"
julia> `grep xylophone $names.txt`
`grep xylophone foo.txt bar.txt baz.txt`
If multiple arrays are inserted into the same word, Julia mimics the behavior generated by the Shell's Descartes product:
julia> names = ["foo","bar","baz"]
3-element ASCIIString Array:
"foo"
"bar"
"baz"
julia> exts = ["aux","log"]
2-element ASCIIString Array:
"aux"
"log"
julia> `rm -f $names.$exts`
`rm -f foo.aux foo.log bar.aux bar.log baz.aux baz.log`
Do not construct temporary array objects, directly interpolate text-based arrays:
julia> `rm -rf $["foo","bar","baz","qux"].$["aux","log","pdf"]`
`rm -rf foo.aux foo.log foo.pdf bar.aux bar.log bar.pdf baz.aux baz.log baz.pdf qux.aux qux.log qux.pdf`
When commands are complex, quotation marks are sometimes required. Let's look at a perl command:
sh$ perl -le '$|=1; for (0..3) { print }'
0
1
2
3
Then look at a command that uses double quotes:
sh$ first="A"
sh$ second="B"
sh$ perl -le '$|=1; print for @ARGV' "1: $first" "2: $second"
1: A
2: B
In general, Julia's inverted quotation marks syntax supports copying and pasting shell commands as they are, and escape, reference, interpolation, and so on work as they are. T he only difference is that the interpolation is integrated into Julia:
julia> `perl -le '$|=1; for (0..3) { print }'`
`perl -le '$|=1; for (0..3) { print }'`
julia> run(ans)
0
1
2
3
julia> first = "A"; second = "B";
julia> `perl -le 'print for @ARGV' "1: $first" "2: $second"`
`perl -le 'print for @ARGV' '1: A' '2: B'`
julia> run(ans)
1: A
2: B
When you need to run the shell command in Julia, try copying and pasting first. Julia displays the command first, where she can check that the interpolation is correct, and then runs the command.
Shell meta-characters, such
&
|
and
>
, are not special characters in Julia's inverted quotation marks syntax.
The pipe character in the inverted quote is simply the text-based | "":
julia> run(`echo hello | sort`)
hello | sort
To construct a pipeline in Julia, you should
|>
Cmd
julia> run(`echo hello` |> `sort`)
hello
Continue with an example:
julia> run(`cut -d: -f3 /etc/passwd` |> `sort -n` |> `tail -n5`)
210
211
212
213
214
It prints the IDs of the five most advanced users of the UNIX system.
cut
sort
and
tail
all run as direct sub-processes of the current
julia
process, and the shell process does not intervene. J
ulia sets up the pipeline herself and connects the file descriptors, which are usually done by the shell. A
s a result, Julia can achieve better control over sub-processes, or it can implement functions that shells cannot. I
t
|>
redirected
stdout
.
Use
.>
to redirect
stderr
.
Julia can run multiple commands in parallel:
julia> run(`echo hello` & `echo world`)
world
hello
The output order is non-determinic. T
wo
echo
start almost at the same time,
stdout
descriptor, which is common to
julia
process. U
sing pipelines, you can pass the output of these processes to other programs:
julia> run(`echo world` & `echo hello` |> `sort`)
hello
world
Let's look at a complex example of using Julia to invoke the perl command:
julia> prefixer(prefix, sleep) = `perl -nle '$|=1; print "'$prefix' ", $_; sleep '$sleep';'`
julia> run(`perl -le '$|=1; for(0..9){ print; sleep 1 }'` |> prefixer("A",2) & prefixer("B",2))
A 0
B 1
A 2
B 3
A 4
B 5
A 6
B 7
A 8
B 9
This is a classic example of a single producer double-parallel consumer:
perl
produces 10 rows from 0 to 9, and two parallel processes consume these results, one with the prefix "A" and the other with the prefix "B". W
e don't know which consumer consumes the first line first, but once it starts, the two processes alternately consume these lines. (
Setting
$|=1
in Perl causes the print expression
stdout
handle first; otherwise the output is cached and printed immediately to the pipeline, resulting in only one consumer process reading.)
)
Take a look at a more complex, multi-step producer-consumer example:
julia> run(`perl -le '$|=1; for(0..9){ print; sleep 1 }'` |>
prefixer("X",3) & prefixer("Y",3) & prefixer("Z",3) |>
prefixer("A",2) & prefixer("B",2))
B Y 0
A Z 1
B X 2
A Y 3
B Z 4
A X 5
B Y 6
A Z 7
B X 8
A Y 9
This example is similar to the previous one, with two steps for consumers alone and different delays for two steps.
It is highly recommended that you try these examples yourself to see how they work.