Many of us are familiar with the package managers in our desktops, such as dnf, apt, yum, and many others. An important part of these package managers is that they are able to find dependencies of software you are trying to install, and then install the dependencies for you. As a user, this makes less work for you, so you don’t have to look up the dependencies and install them manually. As a system administrator, this is a useful time saver.
However, sometimes the dependency resolution can take up valuable CPU cycles and memory on low-end and low-cost devices or machines. Trying to install new software or update can be a difficult task on these low-end machines. What is one to do?
Leverage the cloud: Remote dependency solving
Bringing in the power of the cloud, a group of developers from Red Hat Lab Q are excited to bring a new solution for this problem. Introducing server-side dependency solving, otherwise known as remote dependency solving! Remote dependency solving is an open-source project on GitHub that helps save memory and CPU on those devices with slower hardware.
This project was initiated by Jan Zeleny, and is maintained by Petr Hracek and students of FIT VUT Brno, including Josef Řídký, Michal Ruprich, and Šimon Matěj.
What’s the point? How does it work?
You have a device with low-cost hardware and you’ve recently installed Fedora on it. If you wanted to install a package or set of packages, but the device memory isn’t sufficient, memory could overlap. The device could also unexpectedly lose power or restart during the computing of dependencies. In case of failure or crash, dependencies are not determined properly. Sometimes, we might not have any dependencies at all!
Remote dependency solving solves this problem. Dependencies are computed remotely, as the name implies. This helps save those CPU cycles and memory on the machine by off-loading them to “the cloud”, where dependencies are determined and then sent back to the client.
First, the client sends information about the system and what packages to install. Next, the server computes dependencies and returns a set of packages with correct versions back to the client. Finally, the client installs like normal.
Caching
Another feature of this project is caching. This helps in a case where a second client wants to install the same set of packages and dependencies as another client that the server already computed. This saves computing time and results in a faster overall response to the client.
This is mainly useful for computing farms, where one computer solves dependencies and another one installs the set of packages with correct versions, as was computed before. Red Hat Satellite is an example of this type of usage.
A project like this requires infrastructure, but it is not currently within the scope of the development team. There are plans to work on this in the future.
Testing locally
The package is not yet part of the official Fedora repositories. There is work to get this packaged in Fedora, but the team is not there yet. Therefore, the remote dependency solving software needs to be installed from the GitHub repo.
Setting up
First, clone the project on a computer or embedded device with the following command.
This will clone the project on the device. Before project execution, you will need to install dependencies. Unfortunately, you can’t use remote dependency solving for this part! Run the following command to install all required packages.
Compiling and installing
Next, we need to switch to the build directory, or where we cloned the project. Then, we will need to run the cmake command, like the following example shows.
The command verifies that we have all the required packages for the project. If there is no error, we can build the project using make.
The command creates two binary files called rds-client and rds-server.
Running the remote dependency solving server and client
Once you have the two binary files created, you are ready to start. Open two terminals. One will be for rds-server and the other for rds-client. In order to install, erase, or update packages, the commands have to be executed by the root account.
In the first terminal, run the following command.
In the second terminal, use this command.
What’s going on in the background? On the terminal where the server is running, there will be communication with the rds-client terminal.
Install a package
After completing all previous steps, you are ready to install a package. For the example, rds-client will install emacs. Before installing emacs, make sure to check if it is installed or not. This can be done with the following command.
After confirming emacs is not already installed, run the following command to install the package. You should see a similar output like the one provided in this example.
What does this look like on the server? If you go to check the server terminal window while running the command, you will see output similar to the below excerpt.
After everything finishes, check to make sure emacs is installed.
Erase a package
Maybe you changed your mind and you want to remove a package. We will try erasing emacs from the system for this example. Run the following command to erase the package. You should have similar output in the terminal as in the example.
Again, if you check your server terminal window, the output should look similar to the following block of text.
Finally, check if the package is uninstalled.
Updating the system
Updating the system can be done with one command. Run the following command in the client terminal window.
Looking at the server terminal, you should see a similar output to this.
What’s next?
There is active development on this project and plenty of milestones to hit in the future. For those familiar with some of the more advanced features of DNF, there are plans to integrate those features into Remote Dependency Solving in the near future.
Some of the other items planned for future versions are…
Same commands as DNF
Secure communication between client and server
Package project into Fedora repositories
Cache request on the server-side so that we do not need to repeat dependency solving
Start rds-server as a daemon
Include project into Fedora Infrastructure
Learn more about remote dependency solving
You can read more about remote dependency solving by visiting the project repository on GitHub. Additionally, you can also watch the DevConf 2016 presentation first introducing the project.