Upgrade to Magento 2.4 on Ubuntu (18.04), including MariaDB and ElasticSearch upgrade (single machine)

Magento 2.4 introduces new requirements that can make the upgrade a bit tricky. One of them is that the mysql search engine is deprecated and it must be replaced by elasticsearch BEFORE upgrading. Another one is that mariadb 10.1 is not anymore supported. In this tutorial we will see how to overcome these issues and upgrade successfully.

Disclaimer: due to requirement to have elasticsearch, a minimum of 4GB of RAM is a must! This article is for education purpose only, upgrade is at your own risk.

Magento installation:
If you are interested in installing magento from scratch, you can have a look at my article on how to install a full magento optimized stack (though not compatible with 2,4 since elasticsearch was not yet in the picture, but you can install 2.3.5 this way and upgrade, or wait till I rewrite it :) )
https://medium.com/@vincentteyssier/install-magento-2-3-2-3f0a5b2945c4

Install Elasticsearch

Let’s go:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https
echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list
sudo apt-get update && sudo apt-get install elasticsearch
sudo -i service elasticsearch start
curl -XGET '<host>:9200/_cat/health?v&pretty'

If everything went well the last command should return a nice output like this:

epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks
1519701563 03:19:23  elasticsearch green           1         1      0   0    0    0        0             0

Configure Nginx for elasticsearch

Now we need to create a proxy to use to serve elasticsearch to the webapp. A secure way to do that is detailed here, but the explanation is not very detailed and might be confusing, so let’s make an unsecure version we can fallback to if following this tuto doesn’t work for you

Configure nginx and Elasticsearch

Create the following file and restart nginx:

nano /etc/nginx/conf.d/magento_es_auth.conf
...
Add the following text in the editor, use 8085 or any unused port of your preference
...
server {
   listen 8080;
   location /_cluster/health {
      proxy_pass http://localhost:9200/_cluster/health;
   }
}
...
Exit your editor
...
service nginx restart

Now you can configure Magento to use elasticsearch:

In your admin panel, Click Stores > Settings > Configuration > Catalog > Catalog > Catalog Search.

Elasticsearch Server Hostname: leave localhost

Elasticsearch Server Port: enter the port you configured above (8085 in my example)

That’s it. Save the config, clean your cache, and reindex your site:

bin/magento cache:clean
bin/magento indexer:reindex

Upgrade MariaDB

First thing to do is to create a backup of your DB:

mysqldump -u root -p -x -A > /home/myuser/db_backup.sql

Then stop the database:

sudo systemctl stop mariadb

We now need to uninstall the db:

sudo apt-get remove --auto-remove mariadb-server

Now we need to install MariaDB version compatible with Magento 2.4, i.e. between 10.2 and 10.4. Let’s go for the latest. We need first to configure the right repository:

sudo apt-get install software-properties-common
sudo add-apt-repository 'deb [arch=amd64,arm64,ppc64el] http://sfo1.mirrors.digitalocean.com/mariadb/repo/10.4/ubuntu bionic main'
sudo apt update

Then you can install it with apt:

sudo apt-get install mariadb-server galera-4 mariadb-client libmariadb3 mariadb-backup mariadb-common

Start the DB:

sudo systemctl start mariadb

During the install it will give a warning that your db files are not compatible and that it will back them up. Let it do.

Remember you backed up your db via mysqldump…. good, but if you try to import it you will fail. That’s a known bug listed here: https://jira.mariadb.org/browse/MDEV-22127

To overcome it, it is quite simple, open the dump file with your favorite editor and find the following line:

DROP TABLE IF EXISTS `user`;

Above it, add the following line:

DROP VIEW IF EXISTS `user`;

And in the CREATE line just below, add IF NOT EXISTS so it looks like:

CREATE TABLE IF NOT EXISTS...

That’s it, you fixed it!
Now let’s import it in our DB:

mysql -u root < /home/myuser/db_backup.sql

Success! Finally let’s upgrade your db just to be sure:

sudo mysql_upgrade --force

Ok, we are done.

Upgrade magento

Now that all prerequisites are set, let’s start by backing up your composer file:

cp composer.json composer.json.bak

Update composer and download dependencies, then clean the var folders:

composer require magento/product-community-edition 2.4.0 --no-update
composer update
rm -rf var/cache/* var/page_cache/* var/generation/*

Upgrade Magento itself:

php bin/magento setup:upgrade

If ever you encounter a problem that your db user cannot connect, then you need to recreate it in mariadb:

mysql -u root
GRANT ALL PRIVILEGES ON magentodb.* TO 'yourmagentodbuser'@'localhost' IDENTIFIED BY 'yourmagentodbuserpassword';
FLUSH PRIVILEGES;
exit;

Now I usually recommend to regenerate all files and restart services to make sure you have a clean install (replace www-data by your magento system user and group):

bin/magento setup:upgrade &&
bin/magento setup:di:compile &&
bin/magento setup:static-content:deploy -f &&
bin/magento cache:flush &&
chown -R www-data:www-data ./ && 
service php7.3-fpm restart &&
service nginx restart &&
service varnish restart

That’s it! This was not the easiest upgrade when you have accumulated a bit of legacy, but this is sortable and Magento can still run on a single machine (though 4Gb RAM is a bare minimum now).

Need help with your Magento installation, performances or strategy, I am also providing consulting and implementation services. Contact me by email v.teyssier@gmail.com