tjh.dev
1<chapter xmlns="http://docbook.org/ns/docbook"
2 xmlns:xlink="http://www.w3.org/1999/xlink"
3 xml:id="users-guide-to-the-erlang-infrastructure">
4
5<title>User's Guide to the Erlang Infrastructure</title>
6<section xml:id="build-tools">
7 <title>Build Tools</title>
8 <para>
9 By default Rebar3 wants to manage it's own dependencies. In the
10 normal non-Nix, this is perfectly acceptable. In the Nix world it
11 is not. To support this we have created two versions of rebar3,
12 <literal>rebar3</literal> and <literal>rebar3-open</literal>. The
13 <literal>rebar3</literal> version has been patched to remove the
14 ability to download anything from it. If you are not running it a
15 nix-shell or a nix-build then its probably not going to work for
16 you. <literal>rebar3-open</literal> is the normal, un-modified
17 rebar3. It should work exactly as would any other version of
18 rebar3. Any Erlang package should rely on
19 <literal>rebar3</literal> and thats really what you should be
20 using too.
21 </para>
22</section>
23
24<section xml:id="how-to-install-erlang-packages">
25 <title>How to install Erlang packages</title>
26 <para>
27 Erlang packages are not registered in the top level simply because
28 they are not relevant to the vast majority of Nix users. They are
29 installable using the <literal>erlangPackages</literal> attribute set.
30
31 You can list the avialable packages in the
32 <literal>erlangPackages</literal> with the following command:
33 </para>
34
35 <programlisting>
36$ nix-env -f "<nixpkgs>" -qaP -A erlangPackages
37erlangPackages.esqlite esqlite-0.2.1
38erlangPackages.goldrush goldrush-0.1.7
39erlangPackages.ibrowse ibrowse-4.2.2
40erlangPackages.jiffy jiffy-0.14.5
41erlangPackages.lager lager-3.0.2
42erlangPackages.meck meck-0.8.3
43erlangPackages.rebar3-pc pc-1.1.0
44 </programlisting>
45 <para>
46 To install any of those packages into your profile, refer to them by
47 their attribute path (first column):
48 </para>
49 <programlisting>
50$ nix-env -f "<nixpkgs>" -iA erlangPackages.ibrowse
51 </programlisting>
52 <para>
53 The attribute path of any Erlang packages corresponds to the name
54 of that particular package in Hex or its OTP Application/Release name.
55 </para>
56</section>
57<section xml:id="packaging-erlang-applications">
58 <title>Packaging Erlang Applications</title>
59 <section xml:id="rebar3-packages">
60 <title>Rebar3 Packages</title>
61 <para>
62 There is a Nix functional called
63 <literal>buildRebar3</literal>. We use this function to make a
64 derivation that understands how to build the rebar3 project. For
65 example, the epression we use to build the <link
66 xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
67 project follows.
68 </para>
69 <programlisting>
70{stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
71
72buildRebar3 rec {
73 name = "hex2nix";
74 version = "0.0.1";
75
76 src = fetchFromGitHub {
77 owner = "ericbmerritt";
78 repo = "hex2nix";
79 rev = "${version}";
80 sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
81 };
82
83 erlangDeps = [ ibrowse jsx erlware_commons ];
84}
85 </programlisting>
86 <para>
87 The only visible difference between this derivation and
88 something like <literal>stdenv.mkDerivation</literal> is that we
89 have added <literal>erlangDeps</literal> to the derivation. If
90 you add your Erlang dependencies here they will be correctly
91 handled by the system.
92 </para>
93 <para>
94 If your package needs to compile native code via Rebar's port
95 compilation mechenism. You should add <literal>compilePort =
96 true;</literal> to the derivation.
97 </para>
98 </section>
99
100 <section xml:id="hex-packages">
101 <title>Hex Packages</title>
102 <para>
103 Hex packages are based on Rebar packages. In fact, at the moment
104 we can only compile Hex packages that are buildable with
105 Rebar3. Packages that use Mix and other build systems are not
106 supported. That being said, we know a lot more about Hex and can
107 do more for you.
108 </para>
109 <programlisting>
110{ buildHex }:
111 buildHex {
112 name = "esqlite";
113 version = "0.2.1";
114 sha256 = "1296fn1lz4lz4zqzn4dwc3flgkh0i6n4sydg501faabfbv8d3wkr";
115 compilePort = true;
116}
117 </programlisting>
118 <para>
119 For Hex packages you need to provide the name, the version, and
120 the Sha 256 digest of the package and use
121 <literal>buildHex</literal> to build it. Obviously, the package
122 needs to have already been published to Hex.
123 </para>
124 </section>
125</section>
126<section xml:id="how-to-develop">
127 <title>How to develop</title>
128 <section xml:id="accessing-an-environment">
129 <title>Accessing an Environment</title>
130 <para>
131 Often, all you want to do is be able to access a valid
132 environment that contains a specific package and its
133 dependencies. we can do that with the <literal>env</literal>
134 part of a derivation. For example, lets say we want to access an
135 erlang repl with ibrowse loaded up. We could do the following.
136 </para>
137 <programlisting>
138 ~/w/nixpkgs ❯❯❯ nix-shell -A erlangPackages.ibrowse.env --run "erl"
139 Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
140
141 Eshell V7.0 (abort with ^G)
142 1> m(ibrowse).
143 Module: ibrowse
144 MD5: 3b3e0137d0cbb28070146978a3392945
145 Compiled: January 10 2016, 23:34
146 Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
147 Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
148 debug_info,debug_info,nowarn_shadow_vars,
149 warn_unused_import,warn_unused_vars,warnings_as_errors,
150 {i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
151 Exports:
152 add_config/1 send_req_direct/7
153 all_trace_off/0 set_dest/3
154 code_change/3 set_max_attempts/3
155 get_config_value/1 set_max_pipeline_size/3
156 get_config_value/2 set_max_sessions/3
157 get_metrics/0 show_dest_status/0
158 get_metrics/2 show_dest_status/1
159 handle_call/3 show_dest_status/2
160 handle_cast/2 spawn_link_worker_process/1
161 handle_info/2 spawn_link_worker_process/2
162 init/1 spawn_worker_process/1
163 module_info/0 spawn_worker_process/2
164 module_info/1 start/0
165 rescan_config/0 start_link/0
166 rescan_config/1 stop/0
167 send_req/3 stop_worker_process/1
168 send_req/4 stream_close/1
169 send_req/5 stream_next/1
170 send_req/6 terminate/2
171 send_req_direct/4 trace_off/0
172 send_req_direct/5 trace_off/2
173 send_req_direct/6 trace_on/0
174 trace_on/2
175 ok
176 2>
177 </programlisting>
178 <para>
179 Notice the <literal>-A erlangPackages.ibrowse.env</literal>.That
180 is the key to this functionality.
181 </para>
182 </section>
183 <section xml:id="creating-a-shell">
184 <title>Creating a Shell</title>
185 <para>
186 Getting access to an environment often isn't enough to do real
187 development. Many times we need to create a
188 <literal>shell.nix</literal> file and do our development inside
189 of the environment specified by that file. This file looks a lot
190 like the packageing described above. The main difference is that
191 <literal>src</literal> points to project root and we call the
192 package directly.
193 </para>
194 <programlisting>
195{ pkgs ? import "<nixpkgs"> {} }:
196
197with pkgs;
198
199let
200
201 f = { buildHex, ibrowse, jsx, erlware_commons }:
202 buildHex {
203 name = "hex2nix";
204 version = "0.1.0";
205 src = ./.;
206 erlangDeps = [ ibrowse jsx erlware_commons ];
207 };
208 drv = erlangPackages.callPackage f {};
209
210in
211 drv
212 </programlisting>
213 <section xml:id="building-in-a-shell">
214 <title>Building in a shell</title>
215 <para>
216 Unfortunatly for us users of Nix, Rebar isn't very cooperative
217 with us from the standpoint of building a hermetic
218 environment. When building the rebar3 support we had to do some
219 sneaky things to get it not to go out and pull packages on its
220 own. Also unfortunately, you have to do some of the same things
221 when building a project inside of a Nix shell.
222
223 <orderedlist numeration="arabic">
224 <listitem>
225 <para>Run <literal>rebar3-nix-bootstrap</literal> every time
226 dependencies change</para>
227 </listitem>
228 <listitem>
229 <para>Set Home to the current directory.</para>
230 </listitem>
231 </orderedlist>
232
233 If you do these two things then Rebar will be happy with you. I
234 codify these into a makefile. Forunately, rebar3-nix-bootstrap
235 is idempotent and fairly quick. so you can run it as often as
236 you like.
237 </para>
238 <programlisting>
239# =============================================================================
240# Rules
241# =============================================================================
242.PHONY= all test clean repl shell build test analyze bootstrap
243
244all: test
245
246clean:
247 rm -rf _build
248 rm -rf .cache
249
250repl:
251 nix-shell --run "erl"
252
253shell:
254 nix-shell --run "bash"
255
256bootstrap:
257 nix-shell --pure --run "rebar3-nix-bootstrap"
258
259build: bootstrap
260 nix-shell --pure --run "HOME=$(CURDIR) rebar3 compile"
261
262analyze: bootstrap
263 nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer"
264
265test: bootstrap
266 nix-shell --pure --run "HOME=$(CURDIR) rebar3 do compile,dialyzer,eunit"
267
268 </programlisting>
269 <para>
270 If you add the <literal>shell.nix</literal> as described and
271 user rebar as follows things should simply work.
272 </para>
273 </section>
274</section>
275</section>
276<section xml:id="generating-packages-from-hex-with-hex2nix">
277 <title>Generating Packages from Hex with Hex2Nix</title>
278 <para>
279 Updating the Hex packages requires the use of the
280 <literal>hex2nix</literal> tool. Given the path to the Erlang
281 modules (usually
282 <literal>pkgs/development/erlang-modules</literal>). It will
283 happily dump a file called
284 <literal>hex-packages.nix</literal>. That file will contain all
285 the packages that use a recognized build system in Hex. However,
286 it can't know whether or not all those packages are buildable.
287 </para>
288 <para>
289 To make life easier for our users, it makes good sense to go
290 ahead and attempt to build all those packages and remove the
291 ones that don't build. To do that, simply run the command (in
292 the root of your <literal>nixpkgs</literal> repository). that follows.
293 </para>
294 <programlisting>
295$ nix-build -A erlangPackages
296 </programlisting>
297 <para>
298 That will build every package in
299 <literal>erlangPackages</literal>. Then you can go through and
300 manually remove the ones that fail. Hopefully, someone will
301 improve <literal>hex2nix</literal> in the future to automate
302 that.
303 </para>
304</section>
305</chapter>