How to speed up slow Magento indexer process? Maria DB 10.4

Category Products index taking too much time?

bin/magento index:reset

bin/magento index:reindex

Stuck at category products index …

Here is the solution if you are using MariaDB 10.4

You need to update options/parameters for your db. If using RDS, edit “DB instance parameter group” that you are using and update these 2 values:

optimizer_switch='rowid_filter=off'
optimizer_use_condition_selectivity = 1



After these changes, restart your database. In aws, RDS automatically updates these parameters without any downtime or requirement for restarting. You will see noticeable difference in indexer time.

Reference:

https://devdocs.magento.com/guides/v2.4/performance-best-practices/configuration.html

https://devdocs.magento.com/guides/v2.4/extension-dev-guide/indexer-batch.html

https://magento.stackexchange.com/a/336816/32283

How to export multiple selected tables in PMA phpmyadmin (but not all tables)

Click on export, and choose “custom” export method.

Lets say you want to export first 20 tables only.

Open chrome or browser console [ctrl+shift+j], enter the following:

$("input[name='table_data[]']").slice(0,20).click();


For next 20, use 20,50 as arguments in slice function used above.

How to solve MySQL server gone away error Maria Db update on cPanel and WHM?

This is related to new MariaDb upgrade from 10.1.41 to 10.1.42 and also for servers which were updated from 10.2.27 to 10.2.28

Go to solution ( Special Thanks to @Valetia )

You will see errors in following formats/messages:

  • No file or input found
  • MySQL Server has gone away
  • Connection to MySQL Server failed
  • sqlstate[hy000]: general error: 2013 lost connection to mysql server during query
  • General error: 2006 MySQL server has gone away
  • ERROR 2006 (HY000): MySQL server has gone away
  • ERROR 2013 (HY000): Lost connection to MySQL server during query

Log files (/var/lib/mysql/$hostname.err) will have or can have any of these errors:

  • assertion fail /home/buildbot/buildbot/padding_for_CPACK_RPM_BUILD_SOURCE_DIRS_PREFIX/mariadb-10.2.28/storage/innobase/dict/dict0dict.cc line 1467
  • the resulting row size is greater than maximum allowed size (8126) for a record on index leaf page
  • stack_bottom = 0x0 thread_stack 0x49000 mysys/stacktrace.c:268(my_print_stacktrace)[0x5564e62807bb] sql/signal_handler.cc:209(handle_fatal_signal)[0x5564e5d4b4f5]

Other Symptoms:

/etc/init.d/mysql start
Starting MySQL/etc/init.d/mysql: line 159: kill: (10704) – No such process
[FAILED]


Solution:

yum downgrade MariaDB-* -y
whmapi1 update_updateconf RPMUP=manual UPDATES=manual

(run as root #)

This will downgrade MariaDb installation to previous version
Line 2 will cancel automatic update on WHM/CPanel installations

If you are not using WHM, use line 1 only OR you can downgrade the repository individually too

yum downgrade MariaDB-server MariaDB-common MariaDB-shared MariaDB-client MariaDB-compat MariaDB-devel

References:

How to change attribute dropdown type to multi-select in Magento 2?

Short Answer:

Not possible via Magento2 Admin Backend.

Solution:

You need to update eav_attribute table and edit information about backend_source, frontend_input etc.

Simple query for that is:

UPDATE `eav_attribute` SET `backend_model`="Magento\\\Eav\\\Model\\\Entity\\\Attribute\\\Backend\\\ArrayBackend", `backend_type`="varchar", `frontend_input`="multiselect", `source_model`=NULL WHERE `attribute_id`=YOUR_ATTRIBUTE_ID_INTEGER LIMIT 1

Replace YOUR_ATTRIBUTE_ID_INTEGER with your attribute_id like 355

Reference:

https://stackoverflow.com/a/57701845/2229148

How to wipe all the tables and data in MySQL? Clean whole database

SET FOREIGN_KEY_CHECKS = 0;
SET GROUP_CONCAT_MAX_LEN=32768;
SET @tables = NULL;
SELECT GROUP_CONCAT('`', table_name, '`') INTO @tables
  FROM information_schema.tables
  WHERE table_schema = (SELECT DATABASE());
SELECT IFNULL(@tables,'dummy') INTO @tables;
 
SET @tables = CONCAT('DROP TABLE IF EXISTS ', @tables);
PREPARE stmt FROM @tables;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SET FOREIGN_KEY_CHECKS = 1;

Will not work via phpmyadmin or any script. this has to be run in cmd/terminal

Quick Command

wget https://gist.githubusercontent.com/harshvardhanmalpani/e670a8de7aa81673364dd48f125cb9ac/raw/ce04b319bf1594d148d3346e1474119fe1cd1b3f/flushdb.sql

mysql -hYOUR_DATABASE_HOSTNAME -uYOUR_DATABASE_USER -p YOUR_DATABASE_NAME < flushdb.sql

Source:
https://gist.github.com/harshvardhanmalpani/e670a8de7aa81673364dd48f125cb9ac

How to reverse the ids in a mysql table column without breaking primary key constraint?

Let me explain the problem scenario: Let us assume we have a table with 4 columns in it out of which 1 is PRIMARY column and rest 3 contain some data.

What we want to do is to reverse the primary key IDs for that data keeping the rest of data intact. It is like shifting the first row to end and moving last row to starting.

Initial Data

idnameemailanything
1jacobwhateverhow to reverse
3nathanwhosoeverisnathanids in mysql table column
4jagmohanidontknowjagmohanwithout breaking
8monicaiamsexyprimary key constraint
9batmanidontexistinrealworldi am batman

Data after update

idnameemailanything
9jacobwhatever how to reverse
8nathanwhosoeverisnathan ids in mysql table column
4jagmohanidontknowjagmohan without breaking
3monicaiamsexy primary key constraint
1batman idontexistinrealworld i am batman

Dude, you are just reversing the column, what is so tough in this?

So, this seems simple to reverse an array if isolated by key “id“. But you have to understand that this column is the primary key. So if you run a command to change id for “jacob” to “9“. It will give you error: “Duplicate entry for id 9

So here is my proposed solution, I start with pair of first and last row and then swap them. Then swap second and second last row. and So on…

If total rows are odd, we will be left with 1 row which does not need correction because it will already be the middle row.

If total rows are even, we would swap middle two rows too.

Here is my solution in PHP

$possible_group_id="3";
$ai=12720; //can be any high int which does not exist in column `id` yet
$q="select id,email from table_name where `possible_group_id`=$possible_group_id order by id asc";
$r=mysqli_query($f,$q);
$row_collection=[];
$ct=[];
while($row=mysqli_fetch_row($r))
{
    $row_collection[]=$row;
    $ct[]=$row[0];
}
$pt=array_reverse($ct);
$size=count($row_collection);
for($i=0;$i<$size;$i++)
{
    $row_collection[$i][2]=$pt[$i];
}
echo "start transaction;<br>";
foreach($row_collection as $k=>$v)
{    
    if($k < floor($size/2)){
    echo 'UPDATE `table_name` SET `id`='.$ai.' WHERE `email`="'.$row_collection[$size-$k-1][1].'";<br>';
    echo 'UPDATE `table_name` SET `id`='.$v[2].' WHERE `email`="'.$v[1].'";<br>';
    echo 'UPDATE `table_name` SET `id`='.$row_collection[$k][0].' WHERE `email`="'.$row_collection[$size-$k-1][1].'";<br>';}
    else break;
}
echo "commit;<br>";

It would output this:

start transaction;
UPDATE `table_name` SET `id`=12720 WHERE `email`="idontexistinrealworld ";
UPDATE `table_name` SET `id`=9 WHERE `email`="whatever";
UPDATE `table_name` SET `id`=1 WHERE `email`="idontexistinrealworld ";
UPDATE `table_name` SET `id`=12720 WHERE `email`="iamsexy";
UPDATE `table_name` SET `id`=8 WHERE `email`="whosoeverisnathan";
UPDATE `table_name` SET `id`=3 WHERE `email`="iamsexy";
commit;

Some of the most important SQL commands.

SQL, Structured Query Language, is a programming language designed to manage data stored in relational databases. SQL operates through simple, declarative statements. This keeps data accurate and secure, and it helps maintain the integrity of databases, regardless of size.

Most of the actions you need to perform on a database are done with SQL statements.

SQL commands are grouped into four major categories depending on their functionality:

  • Data Definition Language (DDL)
  • Data Manipulation Language (DML)
  • Transaction Control Language (TCL)
  • Data Control Language (DCL)

COMMANDS:

SELECT

SELECT column_name 
FROM table_name;

SELECT statements are used to fetch data from a database (
extracts data from a database ). Every query will begin with SELECT.

ALTER TABLE

ALTER TABLE table_name 
ADD column_name datatype;

ALTER TABLE lets you add columns to a table in a database.
Modifies a table.

CASE

SELECT column_name,
  CASE
    WHEN condition THEN 'Result_1'
    WHEN condition THEN 'Result_2'
    ELSE 'Result_3'
  END
FROM table_name;

CASE statements are used to create different outputs (usually in the SELECT statement). It is SQL’s way of handling if-then logic.

CREATE TABLE

CREATE TABLE table_name (
  column_1 datatype, 
  column_2 datatype, 
  column_3 datatype
);

CREATE TABLE creates a new table in the database. It allows you to specify the name of the table and the name of each column in the table.

We have an entire article on creating tables in MySQL. https://tutes.in/creating-a-table-in-phpmyadmin-xampp-server/

INSERT

INSERT INTO table_name (column_1, column_2, column_3) 
VALUES (value_1, 'value_2', value_3);

INSERT statements are used to add a new row to a table.

WHERE

SELECT column_name(s)
FROM table_name
WHERE column_name operator value;

WHERE is a clause that filters the result set to include only rows where the specified condition is true.

UPDATE

UPDATE table_name
SET some_column = some_value
WHERE some_column = some_value;

UPDATE statements allow you to edit rows in a table.

AND

SELECT column_name(s)
FROM table_name
WHERE column_1 = value_1
  AND column_2 = value_2;

AND is an operator that combines two conditions. Both conditions must be true for the row to be included in the result set.

OR

SELECT column_name
FROM table_name
WHERE column_name = value_1
   OR column_name = value_2;

OR is an operator that filters the result set to only include rows where either condition is true.

These were the most common and basic SQL commands. There are many more commands in SQL.

Thank You.

Creating a table in PhpMyAdmin Xampp server.

Hello People.

In this post we will discuss that how you can create a table in phpmyadmin xampp server for your php project.

Before creating a table first you must create a database, because without database you can not create a table.

steps to create a table in phpmyadmin.

  • Start your Xampp server and open the following link in your browser.
    localhost/phpmyadmin . Now click on the New button.
  • Give name to your database and hit create button. As shown in the image below.

Our database has been created and now we can start creating the table.

  • Click on the database that you just created in the previous step. Then give name to your table and specify the number of columns under the structure menu.
  • Next you just need to fill your table with the details of your project.

You can also create table using MySql query. For that go to the SQL menu and and write your query. If your query is correct the table will be created. An example is shared below.

CREATE TABLE persons (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    LastName varchar(255) NOT NULL,
    FirstName varchar(255),
    Address varchar(255),
    phone varchar(70) NOT NULL UNIQUE
);  

So, that’s all for now.

Thank You.

Connection timeout with MySQL database.

MySQL disconnects automatically after some time.

If you experience MySQL timeouts, it could be due to heavy or very long MySQL queries.

You can try using mysql_reconnect command before every query, and it should be fine.

MySQL server timeout can occur for many reasons but most commonly it is caused by either an application bug, a network timeout issue, or due to the MySQL server restarting.

These steps could solve the issue. (By setting no time to MySQl.)

  • Edit your my.cnf (MySQL config file)
sudo nano /etc/mysql/my.cnf

  • Add the timeout configuration and adjust it to fit your server.
wait_timeout = 28800
interactive_timeout = 28800
  • Save the changes (CTRL + X , Y , ENTER)
  • Restart MySQL
sudo service mysql restart

The new changes will be applied once it restarts. Hope this will help you.

Thank You.

How to repair a MySQL database?

Hello! Good to see you.

This is going to be a very important article because we are about to discuss  “How to repair a MySQL database?”

Why repair MySQL Database?

Databases can become corrupted for any number of reasons, from software defects to hardware issues.

If this occurs, you can try to repair the database. Wondering how to do it? Just follow this article.

REPAIRING MYSQL DATABASES:

Let’s see how to repair MySQL database through Linux and Windows command line.

Repair MySQL Database through Windows command line.

 

  • Log in to your Windows VPS using Remote Desktop.
  • Go to Start and locate the Command Prompt. Right-click Command Prompt and select Run as Administrator option.
  • At windows command prompt navigate to MySQL bin directory as follows.
    cd C:\Program Files\MySQL\MySQL Server 5.5\bin  //  Replace your MySQL bin directory path here.
  • Run the following command to start the MySQL prompt.
    mysql -u root -p
  • Enter MySQL root password when you are being asked.
  • You will see MySQL prompt appearing. Now, to display all databases, type following command at MySQL prompt:
    show databases;
  • Enter the following command to repair MySQL database.
    mysqlcheck -r [database] // Replace database name with your database name.

Repair MySQL Database through Linux Terminal.

 

  • Login to your Linux VPS using Secure Shell (SSH) and connect to MySQL from the command line.
    mysql -uUsername -p // Replace Username with your username.
  • Enter MySQL user password and hit Enter. You will see MySQL prompt appearing. Now, in order to display all databases, type following command at MySQL prompt.
    show databases;
  • mysqlcheck enables you to check databases without stopping the entire MySQL service. -r argument is used to repair the corrupted tables. mysqlcheck utility efficiently works on both MyISAM and InnoDB database engines. Enter the following command to repair MySQL database.
    mysqlcheck -r [database] // Replace database name with your database name.

 

So, we covered this issue for both Windows and Linux.

Lastly please check our other articles on various topics, that we tried to address.

Thank You.