Home Arch Linux How To Restore Broken Arch Linux To Previous Working State

How To Restore Broken Arch Linux To Previous Working State

By sk
Published: Last Updated on 34.2K views

This brief tutorial describes how to restore broken Arch Linux and its variants like EndeavourOS, Manjaro Linux to previous working state.

The other day, I did a full system upgrade using command "sudo pacman -Syu", and ended up with a broken Arch Linux system.

My Arch Linux refused to boot into graphical mode, and kept displaying the error messages: "dependency failed multi-user system" and "dependency failed for graphical interface".

The only option I had is to login to single user mode and try to rollback the updated packages to their previous versions.

From the single user mode, I tried the following:

  • I tried to install packages from official repositories, but I couldn't. Because my Network card is not recognized in single user mode, so Internet didn't work.
  • The "downgrade" command doesn't work. I have no idea why. So, I couldn't downgrade any packages to their previous versions. It simply displayed an error message "sudo pacman command is not found".
  • So, tried again to downgrade all packages to a specific date. It didn't help either. I got the same error as above i.e. "sudo pacman command is not found".

I was helpless. All I have is a Tablet PC with an Internet connection. After extensive search on Arch Linux wiki and forums, I thought to give a one-last try.

I decided to downgrade all the packages which are previously updated one by one. Because, the "pacman -Syu" command updated many packages, so I couldn't confirm exactly which package is causing the problem.

If you're in this similar situation, this guide might be help you to restore your broken Arch Linux to previous working state.

Restore broken Arch Linux to previous working state

1. First, login in single user mode. To do so, enter "e" when you see the Grub menu.

Arch Linux Grub menu
Grub menu in Arch Linux

2. Then, find the line that starts with word linux:

linux          /boot/vmlinuz-linux  root-UUID=d474f2-e6a2-4cc3-9899-aa98af13  rw  quiet

At the end of the above line, type the following line:

init=/bin/bash

See the following screenshot for your reference.

Edit Grub boot menu entries
Edit Grub boot menu entries

3. Then, press F10 or CTRL+X to continue. After couple seconds, you will be landed into single user mode.

4. Type the following command to mount your root (/) file system in read/write mode.

mount -n -o remount,rw /

5. Now, you need to find when you did the full system upgrade. You can easily find this by looking at your pacman.log file. This file saves all pacman entries you did in the past.

tail -n 200 /var/log/pacman.log | less

Here, -n 200 will display the last 200 lines in your pacman.log file. Because, my pacman.log file has so many entries. 

As you may know, tail command will display the last 10 entries only. So, replace 200 with your own number to go through the pacman.log file. And I piped the output of "tail" command to "less" command to display the results page by page.

6. After going through the pacman.log file, I found the exact time of the full system upgrade. Check the timestamp in the below screenshot (Sorry for the low quality image).

I had run "sudo pacman -Syu" command at 12:06 pm.

Pacman log file in Arch Linux
Pacman log file in Arch Linux

As you can see, "pacman -Syu" command has updated 40+ packages. 

7. Since, I can't use "downgrade" command and downgrade all packages to any specific date, I had no choice but to downgrade all updated packages one by one using "pacman -U" command from the cache. As you already know, pacman saves all downloaded packages in /var/cache/pacman/pkgs/ folder.

8. I noted down all updated packages from the screenshot and downgraded them one by one to earlier version from the cache like below.

pacman -U /var/cache/pacman/pkg/qt5-base-5.9.1-3-x86_64.pkg.tar.xz

The above command will install qt5-base package from the cache folder. It took me nearly half an hour to downgrade all packages.

9. If you cleared the cache folder already, you have no choice. Reinstallation might be the only option. So, I recommend you to keep at least two old versions in the cache, so you can restore them if there are any problems in the new packages.

10. After downgrading all packages, type the following command to apply the changes and start your Arch Linux in to normal mode.

exec /sbin/init

11. Now, I can be able to login to my Arch Linux desktop without any problems. I think I should hold a couple of days and then try to update again when this is fixed.

If you have time and patience, downgrade one package and run "exec /sbin/init" to boot your system into normal mode and check if it solves the problem.

If not, try another package. Repeat this until you find the problematic package. Then simply leave the problematic package and update all other packages.

Some of you may find this method is not so efficient and complicated. But this is how I recovered my broken Arch Linux system.

You might know some other easy ways than this method. If you know any such easy methods, please let me know in the comment section below. I will check and update the guide accordingly.

You May Also Like

11 comments

Lasp94 October 13, 2017 - 4:11 am

Some grep magician would be needed here!!!
Even without grep, I would have copied part of the log, edited with nano and made a little script.. Good tips however!

Reply
steve_dupuis October 14, 2017 - 4:49 am

One of my personal work habits is to always do a backup of the root filesystem before doing kernel or graphics device driver updates. It takes me 10 minutes with Clonezilla (backup and verify), and when the kernel horks I do a 5-minute restore and put an ignore on the update and wait for the next release.
This has saved me much time and much grief over the years.

Reply
SK October 14, 2017 - 12:14 pm

Good idea. Never thought of it.

Reply
pazof October 25, 2018 - 6:34 pm

Just a minor addition – if the packages required for downgrading are for some reason not available in /var/cache/pacman/pkg/ anymore you do not necessarily need to reinstall the whole system, you can always download the packages in question anew from https://archive.archlinux.org/packages/ (along with their respective signature files). You can use a simple wget command to do this, or download them through another device if you are unable to connect to the internet.

Reply
sk October 26, 2018 - 11:04 am

Thank you.

Reply
Nilson October 29, 2019 - 12:35 am

Thanks, man! You saved my life!

Reply
Ron December 13, 2019 - 5:48 am

This has caused me many a grief. Always the graphics setup for my ancient workhorse laptop. I finally abandoned my beloved 4 year old Arch install to Mint.

Reply
Srini R December 31, 2020 - 3:17 am

You saved my day! I ran into a blank boot screen issue after manually upgrading the icu package from icu-67.1-1 to icu-68.2-1 for resolving a dependency required for p3x onenote. Followed this instruction and downgraded the package from cache folder and boom! I’m back into business!

Reply
sk December 31, 2020 - 12:13 pm

Glad I could help. Regards.

Reply
gordonq August 10, 2022 - 5:17 pm

Many thanks!

Reply
adolfo October 3, 2023 - 1:58 am

Life saver! thanks! it helped restore my system after a failed update while on the move, with nothing but the phone and the laptop.

Reply

Leave a Comment

* By using this form you agree with the storage and handling of your data by this website.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. By using this site, we will assume that you're OK with it. Accept Read More