Skip to content

Commit

Permalink
Merge pull request ocaml#3299 from OCamlPro/doc-rehaul
Browse files Browse the repository at this point in the history
Update docs for 2.0.0
  • Loading branch information
AltGr authored Apr 18, 2018
2 parents 81fe930 + ab0b215 commit a454973
Show file tree
Hide file tree
Showing 12 changed files with 501 additions and 591 deletions.
49 changes: 27 additions & 22 deletions doc/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,9 @@ else
JBUILDER_FILE=
endif

BASESRC=../_build/default/src

ifndef OPAM
OPAM = $(JBUILDER) exec -- opam
endif
SRCDIR = $(sort $(foreach x,$(wildcard $(BASESRC)/*/*),$(dir $x)))

TOPICS = $(shell $(OPAM) help topics)
TOPICS_ADMIN = cache filter index lint list upgrade
Expand All @@ -27,14 +24,6 @@ ifndef OPAM_INSTALLER
OPAM_INSTALLER = $(JBUILDER) exec -- opam-installer
endif

SRCEXTDIR = ../src_ext/lib

ifneq ($(wildcard $(SRCEXTDIR)),)
OCAMLDOC = ocamldoc $(patsubst %,-I %,$(SRCDIR) $(SRCEXTDIR) +compiler-libs)
else
OCAMLDOC = ocamlfind ocamldoc $(patsubst %,-package %,$(PACKS) compiler-libs.toplevel) $(patsubst %,-I %,$(SRCDIR))
endif

.PHONY: man html dev-manual pages
all: man dev html pages

Expand All @@ -52,25 +41,41 @@ man:
$(OPAM_INSTALLER) $(HELPFMT) > man/opam-installer.1

man-html: man
rm -rf $@
rm -rf man-html
mkdir -p $@
for f in $(wildcard man/*); do\
man2html -r $$f > man-html/$$(basename $$f .1).html;\
echo '<!DOCTYPE html>' >$@/index.html
echo '<html><head>' >>$@/index.html
echo ' <title>Opam man-pages index</title>' >>$@/index.html
echo '</head><body>' >>$@/index.html
echo '<h1>Opam $(version) man-pages index</h1>' >>$@/index.html
echo '<ul>' >>$@/index.html
for f in man/*; do\
man2html -r $$f | sed 1,2d > $@/$$(basename $$f .1).html;\
echo " <li><a href=\"$$(basename $$f .1).html\">$$(basename $$f .1)</a></li>" >>$@/index.html;\
done
echo '</ul>' >>$@/index.html
echo '</body></html>' >>$@/index.html

dev:
$(MAKE) -C dev-manual

ODOC_PFX = _build/default/_doc

html:
rm -rf html/ocamldoc
mkdir -p html/ocamldoc
rm -rf html
mkdir -p html
cd .. && $(JBUILDER) build @doc
cp ../$(ODOC_PFX)/odoc.css html/
for f in ../$(ODOC_PFX)/*/*.html ../$(ODOC_PFX)/*/*/*.html; do\
mkdir -p $$(dirname html/$${f#../$(ODOC_PFX)/});\
cp $$f html/$${f#../$(ODOC_PFX)/};\
done
sed 's/%{OPAMVERSION}%/'$(version)'/g' index.html > html/index.html
$(OCAMLDOC) $(wildcard $(BASESRC)/*/*.mli) $(wildcard $(BASESRC)/*/*.ml) -html -d html/ocamldoc || true
$(OCAMLDOC) $(wildcard $(BASESRC)/*/*.mli) $(wildcard $(BASESRC)/*/*.ml) -dot -o dependencies.dot || true
tred dependencies.dot | \
dot -Tsvg > \
html/ocamldoc/dependencies.svg \
|| true
# Not to break older links, add manpages to the `ocamldoc` dir
mkdir -p html/ocamldoc
cd html/ocamldoc && for f in ../*/*/index.html; do\
ln -sf $$f $$(basename $$(dirname $$f)).html;\
done

pages/%.html: pages/%.md
omd $^ -o $@
Expand Down
181 changes: 88 additions & 93 deletions doc/index.html

Large diffs are not rendered by default.

109 changes: 109 additions & 0 deletions doc/pages/External_solvers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# External Solvers

Resolving package installations in the presence of dependencies and conflicts is
known to be an
[NP-complete problem](https://hal.archives-ouvertes.fr/file/index/docid/149566/filename/ase.pdf).
Thankfully, a [big effort](http://www.mancoosi.org/) has already been put into
solving it efficiently:

The `opam` package manager is an instance of the approach described in the article "[A modular package manager architecture](http://dl.acm.org/citation.cfm?id=2401012)", which was one of the outcomes of the [Mancoosi](http://www.mancoosi.org) research project. This architecture relies on dependency solvers for package managers, that communicate with the package manager front-end via the [CUDF format](http://www.mancoosi.org/cudf/).

## Installation and compatibility

As of 2.0.0, opam comes with a CUDF solver built-in by default, so unless you
have specifically compiled without it, you shouldn't have to be worried about
installing an external solver. However, these are still supported, and can be
useful in some specific cases. An external solver can be chosen over the
built-in one using the `--solver` command-line argument, the
`$OPAMEXTERNALSOLVER` environment variable, or the `solver:` field in the
`~/.opam/config` file. If no solver was built in or selected, opam will detect
the availability of `aspcud`, `packup` or `mccs` commands on your system and use
one automatically.

The following CUDF solvers have been tested:

- [aspcud](http://www.cs.uni-potsdam.de/wv/aspcud/) (recommended solution until opam 1.2.2)
- [packup](http://sat.inesc-id.pt/~mikolas/sw/packup/)
- [mccs](http://www.i3s.unice.fr/~cpjm/misc/mccs.html) (a modified version of which is now being used as the built-in solver)
- [p2Cudf](https://wiki.eclipse.org/Equinox/p2/CUDFResolver), which can be
downloaded
[here](http://eclipse.org/equinox/p2/p2CUDF/org.eclipse.equinox.p2.cudf-1.14.jar)
and used with the configuration string `java -jar <jarfile-location> -obj
%{criteria}% %{input}% %{output}%`.

These have been developed by a variety of research teams during the
[MISC competitions](http://www.mancoosi.org/misc/) run yearly from 2010 to 2012.

# Specifying user Preferences for the External Solvers

A fundamental distinguishing feature of the `opam` package manager is the fact that it is designed to reuse state-of-the-art dependency solving technology that gives the users the possibility to express their preferences regarding the operations to be performed during an installation, instead of being bound to an hard-coded strategy.
This section provides basic documentation on this feature, and its usage.

## What are user preferences for installations, and why are them important?
When you request the installation of some packages, say p1...pn, `opam` has a lot to do: it needs to look at all the packages already installed on your machine, find all packages available from the repositories, consider your request, and then come up with a set of actions to be performed to satisfy your request.

Unfortunately, there are a lot of assumptions hidden in your mind when you tell `opam` that you want p1...pn installed: should it choose the latest version of the p1...pn? That seems a sensible thing to do, but sometimes installing a recent version of a package p may lead to downgrading or removing another package q, which is something you might not want. What should `opam` do in this case? Remove q to get the latest p, or keep q and get the most recent p that is compatible with it?
Well, the answer is: it depends! It depends on what _you_ really want, and different users may have different points of view.

User preferences, supported by `CUDF`-compatible solvers, are the means you can use to make the assumptions in your mind explicit and known to the solver used by `opam`, so that the actions performed on your machine correspond to your personalised needs.

## How do I express my preferences?

Preferences are expressed using a simple language built by prefixing a little set of combinators with the `-` (minus) or `+` (plus) operators. The most useful combinators are the following ones:

* `new` : the number of new packages
* `changed` : the number of packages modified
* `removed` : the number of packages removed
* `notuptodate` : the number of packages that are not at their latest version

For example, the preference `-removed` tells the solver that among all possible ways of satisfying your request, it should choose one that minimises the number of packages removed.

These combinators can be combined in a comma separated sequence, that is treated in lexicographic order by the solver.

### Default preferences for an upgrade
For example, the preference `-removed,-notuptodate,-changed` tells the solver that after ensuring that removals are minimised, it should look for a solution that minimises also the number of packages wich are not at their latest version, and then reduce the changes to a minimum.

This is close to the default preference setting used by `opam` when you perform an update or an upgrade, and in practice it tries to bring _all_ your packages to the latest version available, as far as this does not implies removing too many packages. It can be set using the environment variable `OPAMUPGRADECRITERIA`, or the [`solver-upgrade-criteria:`](Manual.html#configfield-solver-upgrade-criteria) configuration field.

### Default preferences for an install
When you request to install a (set of) package(s), in general you do not expect to see all your existing packages updated, and this is why in this case it is preferable to use a different value `-removed,-changed,-notuptodate` that tries to minimise changes to the system. It can be set using the environment variable `OPAMCRITERIA`, or the [`solver-criteria:`](Manual.html#configfield-solver-criteria) configuration field.

### Specifying preferences for opam

`opam` allows to specify your criteria on the command line, using the `--criteria` option, that will apply only to the current command.
For example if you are a very conservative user, you might try issueing the following command:
```
opam install --criteria="-removed,-changed" ...
```

This can also be used for some tricks: if for example you want to repair your set of installed packages, you can use the `opam upgrade` command without specifying a preference for newer versions in the criteria (although you may prefer to run `opam upgrade --fixup` in this case):
```
opam upgrade --criteria="-changed"
```

## Yes, there are different versions of the user preference language

The different editions of the MISC competition led to improving the preferences language, by allowing the user progressively more flexibility. Recent solvers give access to a more sophisticated set of preferences, described in [the 2012 MISC competition rules](http://www.mancoosi.org/misc-2012/criteria/).
For example, using `aspcud >=1.8.0`, you could use

`-count(removed),-count(down),-sum(solution,installedsize),-notuptodate(solution),-count(changed)`

to instruct a solver to minimise downgrades, and mininise the installed size, among other criteria.

The default criteria used by opam use a custom CUDF property `version-lag` that
gives a monotonic measure of the "age" of packages, by counting the number of
newer revisions of the package. They can be seen using the `opam config report`
command:

```
# install-criteria -removed,-count[version-lag,request],-count[version-lag,changed],-changed
# upgrade-criteria -removed,-count[version-lag,solution],-new
```

Notice that these criteria are written for the built-in solver which, being
derived from [`mccs`](https://github.com/AltGr/ocaml-mccs), uses a slightly
different syntax for the criteria: the `-sum(subset,property)` criterion should
be written `-count[property,subset]` instead. We also make use of the `request`
subset here, which applies only to the packages that were part of the user
request, and was introduced in aspcud 1.9.0 and is not part of the official mccs
release.
95 changes: 48 additions & 47 deletions doc/pages/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,6 @@
> interested in the more advanced [Tricks](Tricks.html) for specific use-cases
> or advanced users.
#### 🐫 opam fails, trying to reinstall already installed packages at first upgrade ?

It might be that you are still using Ubuntu "utopic", that shipped with a broken
opam package. That shouldn't happen and didn't in any stable opam release. See
[the bug-report on Ubuntu's launchpad](https://bugs.launchpad.net/ubuntu/+source/opam/+bug/1401346)
for the details.

The best fix is to upgrade your [opam](Install.html) using the community
packages.

---

#### 🐫 What is opam for ?

Easily installing, upgrading and managing your OCaml compiler(s), tools and
Expand All @@ -33,19 +21,23 @@ used for different systems using specific repositories (e.g. for the
#### 🐫 How to get, install and upgrade opam ?

See the [install guide](Install.html).
If upgrading, you can bootstrap using `opam install opam-devel`.

If upgrading, you can bootstrap using `opam install opam-devel`, and follow the
instructions.

---

#### 🐫 Where is the manual ?

opam has git-like, hierarchical manpages. Try `opam --help` for a starting point.
opam has git-like, hierarchical
[manpages](https://opam.ocaml.org/doc/2.0/man/opam.html). Try `opam --help` for
a starting point.

Or get started from the [Usage](Usage.html) guide.

If you want to know more about opam packages, see the [Packaging Howto](Packaging.html).

The reference on the internals and file formats is in the [Manual](Manual.html).
The full reference on the internals and file formats is in the [Manual](Manual.html).

You may also want to browse the [library APIs](api/).

Expand All @@ -61,8 +53,12 @@ installations.

If you choose to create "local switches", the installation prefix will be put in
the specified directory with `/_opam/` appended. Nothing else will be changed.
The `opam build` command creates a local switch in the current directory, in the
form of a `_opam/` subdirectory.

Please note, however, that programs you install using opam won't themselves be
bound by any restrictions.

On Linux, and since opam 2.0.0~rc2, package instructions (build, install,
remove) are also run in a sandbox and guaranteed not to affect your system.

---

Expand Down Expand Up @@ -134,6 +130,10 @@ opam switch import <file.export> --switch <new switch>
The file format is human-readable, so you are free to edit the file before doing
the `import` if you need to customise the installation.

You may also want to have a look at the `opam lock` plugin, that can memorise
the precise set of installed dependencies for a local package, and the
associated `opam install DIR --locked` command that can restore them.

---

#### 🐫 I installed a package by hand / used ``ocamlfind remove`` / fiddled with the installed packages and opam is out of sync. Help !
Expand Down Expand Up @@ -161,8 +161,8 @@ there are several ways you can recover:
#### 🐫 What are the minimum requirements ?

1GB of memory should be all you need. It was reported that you may run into
problems with 512MB of RAM and no swap. Of course, software packages themselves
may be more greedy.
problems with 512MB of RAM and no swap. Of course, compiling the packages may
need more.

---

Expand All @@ -173,30 +173,34 @@ using your system package manager (apt-get, yum, pacman, homebrew, etc.) since
they are outside the scope of opam.

opam metadata includes documentation about these external dependencies, on a
variety of systems/distributions, in the form of a `depext:` field. The `depext`
opam plugin can take care of them for you:
variety of systems/distributions, in the form of a
[`depexts:`](https://opam.ocaml.org/doc/2.0/Manual.html#opamfield-depexts)
field. Opam should print the required system dependencies, as documented for
your OS, upon failure, and the `depext` opam plugin can take care of installing
them for you:

```
opam depext <opam-packages>
```

This should install `opam-depext` if needed, check your OS, and prompt to
install the system packages required by your packages or their dependencies,
through your OS's packaging system.
This should install `opam-depext` if needed and prompt to install the system
packages required by your opam packages or their dependencies, through your OS's
packaging system.

If that doesn't work...
* Check for hints printed by the failing package
* Dependencies for your specific system may not be known, but check the output
of `opam list --rec --required-by <package>,<package>... --external`: it will
list dependencies on all known systems and may get you in the right direction.
* Lookup the development packages corresponding to the error in your system's
package repositories.
* Dependencies for your specific system may not be known, but check the output
of `opam list --rec --resolve <package>,<package>... --columns name,depexts:`:
it will list dependencies on all known systems and may get you in the right
direction.

In any of these cases, that's useful information that was missing from the opam
repository: we would really appreciate that you take a minute to save others the
trouble of looking by filling an issue in
[the opam-repository tracker](https://github.com/ocaml/opam-repository/issues),
with your system details, the output of `opam depext --flags`, and the solution,
with your system details, the output of `opam config report`, and the solution,
if you found it. Thanks!

---
Expand All @@ -221,13 +225,10 @@ As a last resort, you can bypass the checksum checks using `--no-checksums`.
#### 🐫 opam is prompting me to install or upgrade packages that I am not interested in, or doesn't install the latest version by default. Why ? What can I do ?

* You can be more explicit in your request (`opam upgrade PACKAGES`, `opam
install 'PACKAGE>=VERSION' PACKAGE...`, etc.)
* Action resolution in a package set is known to be a NP-complete problem; opam
uses state-of-the-art algorithms through an external, dedicated solver: make
sure you have a recent
[external solver installed](Install.html#ExternalSolvers)
* Another benefit of the external solvers is that they allow to be [quite
expressive](Specifying_Solver_Preferences.html) on your expectations.
install 'PACKAGE>=VERSION' PACKAGE...`, etc.). The latest version may not be
available on your system, in this case this will tell you why.
* See how to set [solver preferences](External_solvers.html) that could match your intentions better than the defaults
* Check for pending reinstallations `opam reinstall --list-pending`

---

Expand All @@ -241,15 +242,12 @@ requesting, here is how to find out. We'll suppose you were trying to install
* The above may find a solution by using older version of the packages, in that
case try and force the latest versions thusly: `opam install foo.2.0 bar.1.1`
(you can also use constraints like `'foo>=2.0'`).
* Like in the previous question, make sure you have
[aspcud](http://potassco.sourceforge.net/) or another supported solver
installed, the proposed solutions may not be as accurate without it.

---

#### 🐫 Where do I report Bugs, Issues and Feature Requests?

- Bug reports and feature requests for the opam tool should be reported on
- Bug reports and feature requests for the opam **tool** should be reported on
[opam's issue-tracker](https://github.com/ocaml/opam/issues). Please include the
output of `opam config report` whenever applicable.

Expand All @@ -262,20 +260,23 @@ issue-tracker](https://github.com/ocaml/opam-repository/issues).
insights and evolution of opam internals can discussed on the [opam-devel
mailing-list](http://lists.ocaml.org/listinfo/opam-devel).

- You may also try IRC channel `#opam` on Freenode.
- https://discuss.ocaml.org is a good place for community assistance.

- You may also try IRC channel `#ocaml` on Freenode.

---

#### 🐫 How to link to libraries installed with opam ?

The standard way of doing this is to use
[ocamlfind](https://opam.ocaml.org/packages/ocamlfind), which is
orthogonal to opam: `ocamlfind query <lib>`.

Your libraries are installed to the directory returned by ``opam config var
lib``, which is by default `~/.opam/<switch>/lib`. Note that using `ocamlc`'s
option `-I +dir` will make `dir` relative to `lib/ocaml`, and will only work for
the libraries that ship with the compiler. Also, remember to add the dependency when
[ocamlfind](https://opam.ocaml.org/packages/ocamlfind), which is orthogonal to
opam: `ocamlfind query <lib>`. If you use [dune](https://github.com/ocaml/dune),
this should be completely transparent.

Your libraries are installed to the directory returned by ``opam var lib``,
which is by default `~/.opam/<switch>/lib`. Note that using `ocamlc`'s option
`-I +dir` will make `dir` relative to `lib/ocaml`, and will only work for the
libraries that ship with the compiler. Also, remember to add the dependency when
you package your project !

---
Expand Down
Loading

0 comments on commit a454973

Please sign in to comment.