Found this gem on another website:
SELECT CONCAT('ALTER TABLE ', tbl.TABLE_SCHEMA, '.', tbl.TABLE_NAME, ' CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;') FROM information_schema.TABLES tbl WHERE tbl.TABLE_SCHEMA = '<TABLENAME>';
This query will generate sql statements to alter all tables in a MySQL db to utf8mb4 collation.