You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: runbot/documentation/dynamic_config.md
+31-31Lines changed: 31 additions & 31 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,17 +1,17 @@
1
1
# Dynamic configs
2
2
3
-
Dynamics configs are a way to define how to install, tests, create build or even run script from a json file that can be either defined on the config server side, or in the code in a specified path.
3
+
Dynamic configs are a way to define how to install, test, create builds, or even run scripts from a JSON file that can be defined either on the config server side or in the code at a specified path.
4
4
5
-
Dynamic config can also be extended to define custom behaviour in another repository (extend a blacklist, change behaviour, ...) but also on the config, to define a final layer of configuration.
5
+
Dynamic configs can also be extended to define custom behavior in another repository (extend a blacklist, change behavior, ...) or on the config itself, to define a final layer of configuration.
6
6
7
7
## Example
8
8
9
-
An [example file](../tests/test_build_config_step_dynamic.json) is available and used for testing. It is a simplified version of the parallel tesing used on runbot and will be used as a reference since it is quite complete.
9
+
An [example file](../tests/test_build_config_step_dynamic.json) is available and used for testing. It is a simplified version of the parallel testing used on runbot and will be used as a reference since it is quite complete.
10
10
11
11
## Config Schema
12
12
13
13
### Config root
14
-
The config structure is define this way:
14
+
The config structure is defined as follows:
15
15
```python
16
16
{
17
17
'name': REQUIRED(NAME),
@@ -21,21 +21,21 @@ The config structure is define this way:
21
21
}
22
22
```
23
23
24
-
The steps are the sequence of operation to execute inside a single build. At least one step is required.
24
+
The steps are the sequence of operations to execute inside a single build. At least one step is required.
25
25
26
-
The name is mandatory and should idealy be unique. It is used for display and can also be used as an anchor when using extensions. The NAME can contain alphanumerical character, spaces, _ and -.
26
+
The name is mandatory and should ideally be unique. It is used for display and can also be used as an anchor when using extensions. The NAME can contain alphanumerical characters, spaces, _ and -.
27
27
```json
28
28
"name": "Parallel testing",
29
29
```
30
-
The vars is a dictionary of variables. This can be used to define some values that will be used in multiple places in the config. Variables are currently supported by `description`, `install_modules` and `test_tags`
30
+
The vars key is a dictionary of variables. This can be used to define values that will be used in multiple places in the config. Variables are currently supported by `description`, `install_modules`, and `test_tags`.
31
31
32
32
```json
33
33
"vars": {
34
34
"module_filter": "*,-hw_*"
35
35
},
36
36
```
37
37
38
-
The steps key is a list of step, each step defining a name, a job_type and some parameters specific to the job_type
38
+
The steps key is a list of steps, each step defining a name, a job_type, and some parameters specific to the job_type.
39
39
40
40
```json
41
41
"steps": [
@@ -48,15 +48,15 @@ The steps key is a list of step, each step defining a name, a job_type and some
48
48
]
49
49
```
50
50
51
-
The description can be used to add a custom description to the build, usually used when creating multiple children to differenciate them.
51
+
The description can be used to add a custom description to the build, usually used when creating multiple children to differentiate them.
52
52
53
53
```json
54
54
"description": "Post install tests for **{{test_module_filter}}**",
55
55
```
56
56
57
57
### Config steps
58
58
59
-
The config steps are mainly defined by their `job_type`. The `name` key is also mandatory for display and extensions purposes.
59
+
The config steps are mainly defined by their `job_type`. The `name` key is also mandatory for display and extension purposes.
60
60
61
61
#### Odoo step
62
62
```python
@@ -83,7 +83,7 @@ Both entries will use the value as a runbot module filter, and then passed as th
83
83
```
84
84
In this example, `module_filter` is a variable defined in the vars section.
85
85
86
-
test_tags is used to define which tests to run and will be passed to odoo.
86
+
test_tags is used to define which tests to run and will be passed to odoo.
87
87
88
88
```json
89
89
"test_tags": "-post_install,-/test_lint"
@@ -95,11 +95,11 @@ test_tags is also a dynamic value and can use variables, as well as [filter](#fi
95
95
```
96
96
97
97
98
-
`demo_mode` is used to define the if the flag --with-demo or --without-demo should be passed to odoo during installation. The default value is 'default' meaning that no flag will be passed (behaviour depends on the odoo default behaviour, changed in 18.2)
98
+
`demo_mode` is used to define whether the flag --with-demo or --without-demo should be passed to odoo during installation. The default value is 'default', meaning that no flag will be passed (behavior depends on the odoo default, changed in 18.2).
99
99
100
100
`enable_auto_tags` is True by default and will enable the automatic addition of test-tags when running tests. The automatic test tags are based on build errors.
101
101
102
-
`cpu_limit` is the maximum cpu time that cans be used by the docker. This is also used as a execution time limit for the step. It is mostly useful to avoid having a build stuck and taking a slot for too long. It is not adviced to set this value in the config since it could start to break randomly if the execution time is to close to the limit. This can be set using an [extension](#extensions) on the config in database.
102
+
`cpu_limit` is the maximum CPU time that can be used by the docker. This is also used as an execution time limit for the step. It is mostly useful to avoid having a build stuck and taking a slot for too long. It is not advised to set this value in the config since it could start to break randomly if the execution time is too close to the limit. This can be set using an [extension](#extensions) on the config in the database.
103
103
104
104
#### Create build step
105
105
```python
@@ -111,7 +111,7 @@ test_tags is also a dynamic value and can use variables, as well as [filter](#fi
111
111
}
112
112
```
113
113
114
-
`children` is a list of config, each child will have one of the config defined in the list.
114
+
`children` is a list of configs, each child will use one of the configs defined in the list.
115
115
116
116
```json
117
117
{
@@ -141,9 +141,9 @@ test_tags is also a dynamic value and can use variables, as well as [filter](#fi
141
141
}
142
142
```
143
143
144
-
In this example two children will be created, one testing at_install tests of all modules, the other testing test_lint only.
144
+
In this example, two children will be created: one testing at_install tests of all modules, the other testing test_lint only.
145
145
146
-
It is also possible to define `for_each_vars`, which is a list of variable dictionaries. For each entry in the list, a child will be created for each config in the children list, with the variables overriden by the for_each_vars entry.
146
+
It is also possible to define `for_each_vars`, which is a list of variable dictionaries. For each entry in the list, a child will be created for each config in the children list, with the variables overridden by the for_each_vars entry.
147
147
148
148
```json
149
149
{
@@ -169,7 +169,7 @@ It is also possible to define `for_each_vars`, which is a list of variable dicti
169
169
}
170
170
```
171
171
172
-
In this example, two children will be created, both using the same config, but one will test the mail module, the other the web module. The description is also using the variable to differenciate the two builds.
172
+
In this example, two children will be created, both using the same config, but one will test the mail module and the other the web module. The description also uses the variable to differentiate the two builds.
173
173
174
174
This is how the runbot post install builds are created, using [filters](#filters) to transform module filters into test tags.
175
175
```json
@@ -217,7 +217,7 @@ This job type will restore a database
217
217
By default, it will restore a db "all" in the parent build.
218
218
A `db_name` can be provided to change the database name from "all" to any other value.
219
219
A `build_id` can be provided to restore from another build instead of the parent one.
220
-
A `trigger_id` can be provided to restore from the build created by a specific trigger instead of the parent one by default it will look for such a trigger in the batch from a base branch that was used to create the current batch (base_reference_batch_id). This is mainly usefull for multibuild, restoring a database that is close enough of the current commits automatically when we only want to run the tests. Alternatively, if `use_current_batch` is set to True, the trigger will be searched in the current batch instead of the base_reference_batch_id one. Note that it will most likely fail if executed before the build creating the dump is finished, but can still be usefull for manual triggers when the build will be run manually after the main database is installed, when some model/data changes are needed.
220
+
A `trigger_id` can be provided to restore from the build created by a specific trigger instead of the parent one. By default, it will look for such a trigger in the batch from a base branch that was used to create the current batch (base_reference_batch_id). This is mainly useful for multibuild, restoring a database that is close enough to the current commits automatically when we only want to run the tests. Alternatively, if `use_current_batch` is set to True, the trigger will be searched in the current batch instead of the base_reference_batch_id one. Note that it will most likely fail if executed before the build creating the dump is finished, but can still be useful for manual triggers when the build will be run manually after the main database is installed, when some model/data changes are needed.
221
221
222
222
223
223
### Command steps
@@ -236,7 +236,7 @@ A `trigger_id` can be provided to restore from the build created by a specific t
236
236
237
237
The `command` step will run a custom command inside the odoo container.
238
238
239
-
The `command` is a dynamic value that will be formated with some basic values: `db_name`, `data_dir`, `addons_path`, `exports`, `exports_paths`
239
+
The `command` is a dynamic value that will be formatted with some basic values: `db_name`, `data_dir`, `addons_path`, `exports`, `exports_paths`
Note: db_name is not the same as the db_name passed in parametter, the parameter is actually a suffix where the parameter is the complete dbname (with build dest prefix)
250
+
Note: db_name is not the same as the db_name passed in the parameter; the parameter is actually a suffix, while the parameter is the complete dbname (with build destination prefix).
251
251
252
252
253
253
254
254
## Extensions
255
255
256
-
It is possible to extend a dynamic config by defining an extension either on the config server side, or in another file/repository.
256
+
It is possible to extend a dynamic config by defining an extension either on the config server side or in another file/repository.
257
257
```json
258
258
{
259
259
"extension": true,
@@ -277,8 +277,8 @@ This will alter the variable `module_filter` append `,-l10n_*`, effectively excl
277
277
```
278
278
This will search in all steps for a step named "install all" and set its cpu_limit to 6500.
279
279
280
-
The basic logic of extension is that both the basic and extension structure are explored on all matching entries, until one of the entry of the extension is a command (a two elements list with a command and a value). The command is then applied to the base value.
281
-
When parcouring a list, the matching is done using all keys starting with `@`. In the previous example, the step with name "install all" is matched using the `@name` key. If no key was given, all steps would have been extended.
280
+
The basic logic of extension is that both the base and extension structures are explored on all matching entries, until one of the entries of the extension is a command (a two-element list with a command and a value). The command is then applied to the base value.
281
+
When traversing a list, the matching is done using all keys starting with `@`. In the previous example, the step with name "install all" is matched using the `@name` key. If no key was given, all steps would have been extended.
282
282
283
283
Currently available commands are:
284
284
@@ -288,32 +288,32 @@ Currently available commands are:
288
288
289
289
## Module selection
290
290
291
-
The runbot module selection works by parsing all filters/selectors (commaseparated list) in the given order, each element adding or removing modules from the selection.
291
+
The runbot module selection works by parsing all filters/selectors (comma-separated list) in the given order, each element adding or removing modules from the selection.
292
292
293
293
The basic selector is a fnmatch on a module name.
294
294
295
-
Selector can be prefixed with a `-` to exclude modules matching the selector.
295
+
Selectors can be prefixed with a `-` to exclude modules matching the selector.
296
296
297
297
If not prefixed with a `-`, the selector will add all available modules matching the selector to the selection.
298
298
299
299
The first selector is usually `*`, meaning to select all modules, or '-*', to ensure that we start with an empty list.
300
300
301
301
`*,-hw_*` means to select all modules except those starting with `hw_`.
302
302
`*,-*l10n_*,test_l10n*` means to select all modules except those containing `l10n_`, but still includes modules starting with `test_l10n`.
303
-
`blacklisted_module` will force to include a `blacklisted_module` even if it is blacklisted. Since the selection is not starting with `*`, only module not blacklisted on the repo will be added by default. Note that once a `*` is added at some point, the repo blacklist is completely ignored.
303
+
`blacklisted_module` will force inclusion of a `blacklisted_module` even if it is blacklisted. Since the selection is not starting with `*`, only modules not blacklisted on the repo will be added by default. Note that once a `*` is added at some point, the repo blacklist is completely ignored.
304
304
305
-
Additionally, some filters cans also be used to filter the current list of modules using a range based on alphanumeric sorting of module names:
306
-
`[!]<m1> -> [!]<m1>`
305
+
Additionally, some filters can also be used to filter the current list of modules using a range based on alphanumeric sorting of module names:
306
+
`[!]<m1> -> [!]<m2>`
307
307
308
-
The -> defines a range, selecting all module between m1 and m2 inclusively. If m1 is omitted, the range starts at the beginning of the list, if m2 is omitted, the range ends at the end of the list.
309
-
The ! negates the module, meaning that modules itself will be excluded from the selection.
308
+
The -> defines a range, selecting all modules between m1 and m2 inclusively. If m1 is omitted, the range starts at the beginning of the list; if m2 is omitted, the range ends at the end of the list.
309
+
The ! negates the module, meaning that the module itself will be excluded from the selection.
310
310
311
311
`-> !mail` will keep only modules that are before `mail` in alphanumeric order, excluding mail.
312
312
`mail -> !web` will keep only modules between mail and web, excluding web but including mail.
313
313
`web -> web` will keep web.
314
314
`!web ->` will keep only modules after web, excluding web.
315
315
316
-
Note that using `web` instead of `web->web` would include web, but also all modules before and after web, effectively selecting all modules. An equivalent solution could be to do`-*,web`
316
+
Note that using `web` instead of `web->web` would include web, but also all modules before and after web, effectively selecting all modules. An equivalent solution could be to use`-*,web`.
0 commit comments