This brief tutorial describes how to restore broken Arch Linux and its variants like EndeavourOS, Manjaro Linux to previous working state.
The other day, I did a full system upgrade using command
"sudo pacman -Syu", and ended up with a broken Arch Linux system.
My Arch Linux refused to boot into graphical mode, and kept displaying the error messages:
"dependency failed multi-user system" and
"dependency failed for graphical interface".
The only option I had is to login to single user mode and try to rollback the updated packages to their previous versions.
From the single user mode, I tried the following:
- I tried to install packages from official repositories, but I couldn't. Because my Network card is not recognized in single user mode, so Internet didn't work.
- The "downgrade" command doesn't work. I have no idea why. So, I couldn't downgrade any packages to their previous versions. It simply displayed an error message
"sudo pacman command is not found".
- So, tried again to downgrade all packages to a specific date. It didn't help either. I got the same error as above i.e.
"sudo pacman command is not found".
I was helpless. All I have is a Tablet PC with an Internet connection. After extensive search on Arch Linux wiki and forums, I thought to give a one-last try.
I decided to downgrade all the packages which are previously updated one by one. Because, the
"pacman -Syu" command updated many packages, so I couldn't confirm exactly which package is causing the problem.
If you're in this similar situation, this guide might be help you to restore your broken Arch Linux to previous working state.
Restore broken Arch Linux to previous working state
1. First, login in single user mode. To do so, enter
"e" when you see the Grub menu.
2. Then, find the line that starts with word
linux /boot/vmlinuz-linux root-UUID=d474f2-e6a2-4cc3-9899-aa98af13 rw quiet
At the end of the above line, type the following line:
See the following screenshot for your reference.
3. Then, press
CTRL+X to continue. After couple seconds, you will be landed into single user mode.
4. Type the following command to mount your
root (/) file system in read/write mode.
mount -n -o remount,rw /
5. Now, you need to find when you did the full system upgrade. You can easily find this by looking at your
pacman.log file. This file saves all pacman entries you did in the past.
tail -n 200 /var/log/pacman.log | less
-n 200 will display the last
200 lines in your
pacman.log file. Because, my
pacman.log file has so many entries.
As you may know, tail command will display the last 10 entries only. So, replace
200 with your own number to go through the
pacman.log file. And I piped the output of
"tail" command to
"less" command to display the results page by page.
6. After going through the pacman.log file, I found the exact time of the full system upgrade. Check the timestamp in the below screenshot (Sorry for the low quality image).
I had run
"sudo pacman -Syu" command at 12:06 pm.
As you can see,
"pacman -Syu" command has updated
7. Since, I can't use
"downgrade" command and downgrade all packages to any specific date, I had no choice but to downgrade all updated packages one by one using
"pacman -U" command from the cache. As you already know,
pacman saves all downloaded packages in
8. I noted down all updated packages from the screenshot and downgraded them one by one to earlier version from the cache like below.
pacman -U /var/cache/pacman/pkg/qt5-base-5.9.1-3-x86_64.pkg.tar.xz
The above command will install
qt5-base package from the cache folder. It took me nearly half an hour to downgrade all packages.
9. If you cleared the cache folder already, you have no choice. Reinstallation might be the only option. So, I recommend you to keep at least two old versions in the cache, so you can restore them if there are any problems in the new packages.
10. After downgrading all packages, type the following command to apply the changes and start your Arch Linux in to normal mode.
11. Now, I can be able to login to my Arch Linux desktop without any problems. I think I should hold a couple of days and then try to update again when this is fixed.
If you have time and patience, downgrade one package and run
"exec /sbin/init" to boot your system into normal mode and check if it solves the problem.
If not, try another package. Repeat this until you find the problematic package. Then simply leave the problematic package and update all other packages.
Some of you may find this method is not so efficient and complicated. But this is how I recovered my broken Arch Linux system.
You might know some other easy ways than this method. If you know any such easy methods, please let me know in the comment section below. I will check and update the guide accordingly.
Other useful Arch Linux tutorials on this blog:
- Enable Parallel Downloading In Pacman In Arch Linux
- Retrieve Latest Mirror List Using Reflector In Arch Linux
- The Recommended Way To Clean The Package Cache In Arch Linux
- How To Ignore A Package From Being Upgraded In Arch Linux
- How To Fix “pacman: error while loading shared libraries” Error In Arch Linux
Some grep magician would be needed here!!!
Even without grep, I would have copied part of the log, edited with nano and made a little script.. Good tips however!
One of my personal work habits is to always do a backup of the root filesystem before doing kernel or graphics device driver updates. It takes me 10 minutes with Clonezilla (backup and verify), and when the kernel horks I do a 5-minute restore and put an ignore on the update and wait for the next release.
This has saved me much time and much grief over the years.
Good idea. Never thought of it.
Just a minor addition – if the packages required for downgrading are for some reason not available in /var/cache/pacman/pkg/ anymore you do not necessarily need to reinstall the whole system, you can always download the packages in question anew from https://archive.archlinux.org/packages/ (along with their respective signature files). You can use a simple wget command to do this, or download them through another device if you are unable to connect to the internet.
Thanks, man! You saved my life!
This has caused me many a grief. Always the graphics setup for my ancient workhorse laptop. I finally abandoned my beloved 4 year old Arch install to Mint.
You saved my day! I ran into a blank boot screen issue after manually upgrading the icu package from icu-67.1-1 to icu-68.2-1 for resolving a dependency required for p3x onenote. Followed this instruction and downgraded the package from cache folder and boom! I’m back into business!
Glad I could help. Regards.