Appendix A. Frequently Asked Questions About MySQL 5.0
Prev |  | Next

## Appendix A. Frequently Asked Questions About MySQL 5.0

## A.1. MySQL 5.0 FAQ — General

Questions

- 26.1.1: When did MySQL 5.0 become production-ready (GA)?

- 26.1.2: Can MySQL 5.0 do subqueries?

- 26.1.3: Can MySQL 5.0 peform multi-table inserts, updates, and deletes?

- 26.1.4: Does MySQL 5.0 have a Query Cache? Does it work on Server, Instance or Database?

- 26.1.5: Does MySQL 5.0 have Sequences?

- 26.1.6: Does MySQL 5.0 have a `NOW()` function with fractions of seconds?

- 26.1.7: Does MySQL 5.0 work with multi-core processors?

- 26.1.8: Is there a hot backup tool for MyISAM like InnoDB Hot Backup?

- 26.1.9: Have there been there any improvements in error reporting when foreign keys fail? Does MySQL now report which column and reference failed?

- 26.1.10: Can MySQL 5.0 perform ACID transactions?

Questions and Answers

26.1.1: When did MySQL 5.0 become production-ready (GA)?

MySQL 5.0.15 was released for production use on 19 October 2005. We are now working on MySQL 5.1, which is currently in beta.

26.1.2: Can MySQL 5.0 do subqueries?

Yes. See Section 13.2.8, “Subquery Syntax”.

26.1.3: Can MySQL 5.0 peform multi-table inserts, updates, and deletes?

Yes. For the syntax required to perform multi-table updates, see Section 13.2.10, “`UPDATE` Syntax”; for that required to perform multi-table deletes, see Section 13.2.1, “`DELETE` Syntax”.

A multi-table insert can be accomplished using a trigger whose `FOR EACH ROW` clause contains multiple `INSERT` statements within a `BEGIN ... END` block. See Section 18.3, “Using Triggers”.

26.1.4: Does MySQL 5.0 have a Query Cache? Does it work on Server, Instance or Database?

Yes. The query cache operates on the server level, caching complete result sets matched with the original query string. If an exactly identical query is made (which often happens, particularly in web applications), no parsing or execution is necessary; the result is sent directly from the cache. Various tuning options are available. See Section 5.14, “The MySQL Query Cache”.

26.1.5: Does MySQL 5.0 have Sequences?

No. However, MySQL has an `AUTO_INCREMENT` system, which in MySQL 5.0 can also handle inserts in a multi-master replication setup. With the `--auto-increment-increment` and `--auto-increment-offset` startup options, you can set each server to generate auto-increment values that don't conflict with other servers. The `--auto-increment-increment` value should be greater than the number of servers, and each server should have a unique offset.

26.1.6: Does MySQL 5.0 have a `NOW()` function with fractions of seconds?

No. This is on the MySQL roadmap as a “rolling feature”. This means that it is not a flagship feature, but will be implemented, development time permitting. Specific customer demand may change this scheduling.

However, MySQL does parse time strings with a fractional component. See Section 11.3.2, “The `TIME` Type”.

26.1.7: Does MySQL 5.0 work with multi-core processors?

Yes. MySQL is fully multi-threaded, and will make use of multiple CPUs, provided that the operating system supports them.

26.1.8: Is there a hot backup tool for MyISAM like InnoDB Hot Backup?

This is currently under development for a future MySQL release.

26.1.9: Have there been there any improvements in error reporting when foreign keys fail? Does MySQL now report which column and reference failed?

The foreign key support in `InnoDB` has seen improvements in each major version of MySQL. Foreign key support generic to all storage engines is scheduled for MySQL 5.2; this should resolve any inadequacies in the current storage engine specific implementation.

26.1.10: Can MySQL 5.0 perform ACID transactions?

Yes. All current MySQL versions support transactions. The `InnoDB` storage engine offers full ACID transactions with row-level locking, multi-versioning, non-locking repeatable reads, and all four SQL standard isolation levels.

The `NDB` storage engine supports the `READ COMMITTED` transaction isolation level only.

## A.2. MySQL 5.0 FAQ — Storage Engines

Questions

- 26.2.1: Where can I obtain complete documentation for MySQL storage engines?

- 26.2.2: Are there any new storage engines in MySQL 5.0?

- 26.2.3: Have any storage engines been removed in MySQL 5.0?

- 26.2.4: What are the unique benefits of the `ARCHIVE` storage engine?

- 26.2.5: Do the new features in MySQL 5.0 apply to all storage engines?

Questions and Answers

26.2.1: Where can I obtain complete documentation for MySQL storage engines?

See Chapter 14, Storage Engines and Table Types. That chapter contains information about all MySQL storage engines except for the `NDB` storage engine used for MySQL Cluster; `NDB` is covered in Chapter 15, MySQL Cluster.

26.2.2: Are there any new storage engines in MySQL 5.0?

Yes. The `FEDERATED` storage engine, new in MySQL 5.0, allows the server to access tables on other (remote) servers. See Section 14.7, “The `FEDERATED` Storage Engine”.

26.2.3: Have any storage engines been removed in MySQL 5.0?

Yes. MySQL 5.0 no longer supports the `ISAM` storage engine. If you have any existing `ISAM` tables from previous versions of MySQL, you should convert these to `MyISAM` before upgrading to MySQL 5.0.

26.2.4: What are the unique benefits of the `ARCHIVE` storage engine?

The `ARCHIVE` storage engine is ideally suited for storing large amounts of data without indexes; it has a very small footprint, and performs selects using table scans. See Section 14.8, “The `ARCHIVE` Storage Engine”, for details.

26.2.5: Do the new features in MySQL 5.0 apply to all storage engines?

The general new features such as views, stored procedures, triggers, `INFORMATION_SCHEMA`, precision math (`DECIMAL` column type), and the `BIT` column type, apply to all storage engines. There are also additions and changes for specific storage engines.

## A.3. MySQL 5.0 FAQ — Server SQL Mode

Questions

- 26.3.1: What are server SQL modes?

- 26.3.2: How many server SQL modes are there?

- 26.3.3: How do you determine the server SQL mode?

- 26.3.4: Is the mode dependent on the database or connection?

- 26.3.5: Can the rules for strict mode be extended?

- 26.3.6: Does strict mode impact performance?

- 26.3.7: What is the default server SQL mode when My SQL 5.0 is installed?

Questions and Answers

26.3.1: What are server SQL modes?

Server SQL modes define what SQL syntax MySQL should support and what kind of data validation checks it should perform. This makes it easier to use MySQL in different environments and to use MySQL together with other database servers. The MySQL Server apply these modes individually to different clients. For more information, see Section 5.2.6, “SQL Modes”.

26.3.2: How many server SQL modes are there?

Each mode can be independently switched on and off. See Section 5.2.6, “SQL Modes”, for a complete list of available modes.

26.3.3: How do you determine the server SQL mode?

You can set the default SQL mode (for mysqld startup) with the `--sql-mode` option. Using the statement `SET [SESSION|GLOBAL] sql_mode='`modes`'`, you can change the settings from within a connection, either locally to the connection, or to take effect globally. You can retrieve the current mode by issuing a `SELECT @@sql_mode` statement.

26.3.4: Is the mode dependent on the database or connection?

A mode is not linked to a particular database. Modes can be set locally to the session (connection), or globally for the server. you can change these settings using `SET [SESSION|GLOBAL] sql_mode='`modes`'`.

26.3.5: Can the rules for strict mode be extended?

When we refer to strict mode, we mean a mode where at least one of the modes `TRADITIONAL`, `STRICT_TRANS_TABLES`, or `STRICT_ALL_TABLES` is enabled. Options can be combined, so you can add additional restrictions to a mode. See Section 5.2.6, “SQL Modes”, for more information.

26.3.6: Does strict mode impact performance?

The intensive validation of input data that some settings requires more time than if the validation is not done. While the performance impact is not that great, if you do not require such validation (perhaps your application already handles all of this), then MySQL gives you the option of leaving strict mode disabled. However — if you do require it — strict mode can provide such validation.

26.3.7: What is the default server SQL mode when My SQL 5.0 is installed?

By default, no special modes are enabled. See Section 5.2.6, “SQL Modes”, for information about all available modes and MySQL's default behavior.

## A.4. MySQL 5.0 FAQ — Stored Procedures

Questions

- 26.4.1: Does MySQL 5.0 support stored procedures?

- 26.4.2: Where can I find documentation for MySQL stored procedures and stored functions?

- 26.4.3: Is there a discussion forum for MySQL stored procedures?

- 26.4.4: Where can I find the ANSI SQL 2003 specification for stored procedures?

- 26.4.5: How do you manage stored routines?

- 26.4.6: Is there a way to view all stored procedures and stored functions in a given database?

- 26.4.7: Where are stored procedures stored?

- 26.4.8: Is it possible to group stored procedures or stored functions into packages?

- 26.4.9: Can a stored procedure call another stored procedure?

- 26.4.10: Can a stored procedure call a trigger?

- 26.4.11: Can a stored procedure access tables?

- 26.4.12: Do stored procedures have a statement for raising application errors?

- 26.4.13: Do stored procedures provide exception handling?

- 26.4.14: Can MySQL 5.0 stored routines return result sets?

- 26.4.15: Is `WITH RECOMPILE` supported for stored procedures?

- 26.4.16: Is there a MySQL equivalent to using `mod_plsql` as a gateway on Apache to talk directly to a stored procedure in the database?

- 26.4.17: Can I pass an array as input to a stored procedure?

- 26.4.18: Can I pass a cursor as an `IN` parameter to a stored procedure?

- 26.4.19: Can I return a cursor as an `OUT` parameter from a stored procedure?

- 26.4.20: Can I print out a variable's value within a stored routine for debugging purposes?

- 26.4.21: Can I commit or roll back transactions inside a stored procedure?

Questions and Answers

26.4.1: Does MySQL 5.0 support stored procedures?

Yes. MySQL 5.0 supports two types of stored routines — stored procedures and stored functions.

26.4.2: Where can I find documentation for MySQL stored procedures and stored functions?

See Chapter 17, Stored Procedures and Functions.

26.4.3: Is there a discussion forum for MySQL stored procedures?

Yes. See http://forums.mysql.com/list.php?98.

26.4.4: Where can I find the ANSI SQL 2003 specification for stored procedures?

Unfortunately, the official specifications are not freely available (ANSI makes them available for purchase). However, there are books — such as SQL-99 Complete, Really by Peter Gulutzan and Trudy Pelzer — which give a comprehensive overview of the standard, including coverage of stored procedures.

26.4.5: How do you manage stored routines?

It is always good practice to use a clear naming scheme for your stored routines. You can manage stored procedures with `CREATE [FUNCTION|PROCEDURE]`, `ALTER [FUNCTION|PROCEDURE]`, `DROP [FUNCTION|PROCEDURE]`, and `SHOW CREATE [FUNCTION|PROCEDURE]`. You can obtain information about existing stored procedures using the `ROUTINES` table in the `INFORMATION_SCHEMA` database (see Section 20.14, “The `INFORMATION_SCHEMA ROUTINES` Table”).

26.4.6: Is there a way to view all stored procedures and stored functions in a given database?

Yes. For a database named `dbname`, use this query on the `INFORMATION_SCHEMA.ROUTINES` table:

    SELECT ROUTINE_TYPE, ROUTINE_NAME
        FROM INFORMATION_SCHEMA.ROUTINES
        WHERE ROUTINE_SCHEMA='dbname';

For more information, see Section 20.14, “The `INFORMATION_SCHEMA ROUTINES` Table”.

The body of a stored routine can be viewed using `SHOW CREATE FUNCTION` (for a stored function) or `SHOW CREATE PROCEDURE` (for a stored procedure). See Section 13.5.4.5, “`SHOW CREATE PROCEDURE` and `SHOW CREATE FUNCTION` Syntax”, for more information.

26.4.7: Where are stored procedures stored?

In the `proc` table of the `mysql` system database. However, you should not access the tables in the system database directly. Instead, use `SHOW CREATE FUNCTION` to obtain information about stored functions, and `SHOW CREATE PROCEDURE` to obtain information about stored procedures. See Section 13.5.4.5, “`SHOW CREATE PROCEDURE` and `SHOW CREATE FUNCTION` Syntax”, for more information about these statements.

You can also query the `ROUTINES` table in the `INFORMATION_SCHEMA` database — see Section 20.14, “The `INFORMATION_SCHEMA ROUTINES` Table”, for information about this table.

26.4.8: Is it possible to group stored procedures or stored functions into packages?

No. This is not supported in MySQL 5.0.

26.4.9: Can a stored procedure call another stored procedure?

Yes.

26.4.10: Can a stored procedure call a trigger?

A stored procedure can execute an SQL statement, such as an `UPDATE`, that causes a trigger to fire.

26.4.11: Can a stored procedure access tables?

Yes. A stored procedure can access one or more tables as required.

26.4.12: Do stored procedures have a statement for raising application errors?

Not in MySQL 5.0. We intend to implement the SQL standard `SIGNAL` and `RESIGNAL` statements in a future MySQL release.

26.4.13: Do stored procedures provide exception handling?

MySQL implements `HANDLER` definitions according to the SQL standard. See Section 17.2.8.2, “`DECLARE` Handlers”, for details.

26.4.14: Can MySQL 5.0 stored routines return result sets?

Stored procedures can, but stored functions cannot. If you perform an ordinary `SELECT` inside a stored procedure, the result set is returned directly to the client. You need to use the MySQL 4.1 (or above) client-server protocol for this to work. This means that — for instance — in PHP, you need to use the `mysqli` extension rather than the old `mysql` extension.

26.4.15: Is `WITH RECOMPILE` supported for stored procedures?

Not in MySQL 5.0.

26.4.16: Is there a MySQL equivalent to using `mod_plsql` as a gateway on Apache to talk directly to a stored procedure in the database?

There is no equivalent in MySQL 5.0.

26.4.17: Can I pass an array as input to a stored procedure?

Not in MySQL 5.0.

26.4.18: Can I pass a cursor as an `IN` parameter to a stored procedure?

In MySQL 5.0, cursors are available inside stored procedures only.

26.4.19: Can I return a cursor as an `OUT` parameter from a stored procedure?

In MySQL 5.0, cursors are available inside stored procedures only. However, if you do not open a cursor on a `SELECT`, the result will be sent directly to the client. You can also `SELECT INTO` variables. See Section 13.2.7, “`SELECT` Syntax”.

26.4.20: Can I print out a variable's value within a stored routine for debugging purposes?

Yes, you can do this in a stored procedure, but not in a stored function. If you perform an ordinary `SELECT` inside a stored procedure, the result set is returned directly to the client. You will need to use the MySQL 4.1 (or above) client-server protocol for this to work. This means that — for instance — in PHP, you need to use the `mysqli` extension rather than the old `mysql` extension.

26.4.21: Can I commit or roll back transactions inside a stored procedure?

Yes. However, you cannot perform transactional operations within a stored function.

## A.5. MySQL 5.0 FAQ — Triggers

Questions

- 26.5.1: Where can I find the documentation for MySQL 5.0 triggers?

- 26.5.2: Is there a discussion forum for MySQL Triggers?

- 26.5.3: Does MySQL 5.0 have statement-level or row-level triggers?

- 26.5.4: Are there any default triggers?

- 26.5.5: How are triggers managed in MySQL?

- 26.5.6: Is there a way to view all triggers in a given database?

- 26.5.7: Where are triggers stored?

- 26.5.8: Can a trigger call a stored procedure?

- 26.5.9: Can triggers access tables?

- 26.5.10: Can triggers call an external application through a UDF?

- 26.5.11: Is possible for a trigger to update tables on a remote server?

Questions and Answers

26.5.1: Where can I find the documentation for MySQL 5.0 triggers?

See Chapter 18, Triggers.

26.5.2: Is there a discussion forum for MySQL Triggers?

Yes. It is available at http://forums.mysql.com/list.php?99.

26.5.3: Does MySQL 5.0 have statement-level or row-level triggers?

In MySQL 5.0, all triggers are `FOR EACH ROW` — that is, the trigger is activated for each row that is inserted, updated, or deleted. MySQL 5.0 does not support triggers using `FOR EACH STATEMENT`.

26.5.4: Are there any default triggers?

Not explicitly. MySQL does have specific special behavior for some `TIMESTAMP` columns, as well as for columns which are defined using `AUTO_INCREMENT`.

26.5.5: How are triggers managed in MySQL?

In MySQL 5.0, triggers can be created using the `CREATE TRIGGER` statement, and dropped using `DROP TRIGGER`. See Section 18.1, “`CREATE TRIGGER` Syntax”, and Section 18.2, “`DROP TRIGGER` Syntax”, for more about these statements.

Information about triggers can be obtained by querying the `INFORMATION_SCHEMA.TRIGGERS` table. See Section 20.16, “The `INFORMATION_SCHEMA TRIGGERS` Table”.

26.5.6: Is there a way to view all triggers in a given database?

Yes. You can obtain a listing of all triggers defined on database `dbname` using a query on the INFORMATION_SCHEMA.TRIGGERS table such as the one shown here:

    SELECT TRIGGER_NAME, EVENT_MANIPULATION, EVENT_OBJECT_TABLE, ACTION_STATEMENT
        FROM INFORMATION_SCHEMA.TRIGGERS
        WHERE TRIGGER_SCHEMA='dbname';

For more information about this table, see Section 20.16, “The `INFORMATION_SCHEMA TRIGGERS` Table”.

You can also use the `SHOW TRIGGERS` statement, which is specific to MySQL. See Section 13.5.4.25, “`SHOW TRIGGERS` Syntax”.

26.5.7: Where are triggers stored?

Triggers are currently stored in `.TRG` files, with one such file one per table. In other words, a trigger belongs to a table.

In the future, we plan to change this so that trigger information will be included in the `.FRM` file that defines the structure of the table. We also plan to make triggers database-level objects — rather than table-level objects as they are now — to bring them into compliance with the SQL standard.

26.5.8: Can a trigger call a stored procedure?

Yes.

26.5.9: Can triggers access tables?

A trigger can access both old and new data in its own table. Through a stored procedure, or a multi-table update or delete statement, a trigger can also affect other tables.

26.5.10: Can triggers call an external application through a UDF?

No, not at present.

26.5.11: Is possible for a trigger to update tables on a remote server?

Yes. A table on a remote server could be updated using the `FEDERATED` storage engine. (See Section 14.7, “The `FEDERATED` Storage Engine”).

## A.6. MySQL 5.0 FAQ — Stored Routines, Triggers, and Replication

Questions

- 26.6.1: Do MySQL 5.0 stored procedures and functions work with replication?

- 26.6.2: Are stored procedures and functions created on a master server replicated to a slave?

- 26.6.3: How are actions that take place inside stored procedures and functions replicated?

- 26.6.4: Are there special security requirements for using stored procedures and functions together with replication?

- 26.6.5: What limitations exist for replicating stored procedure and function actions?

- 26.6.6: Do the preceding limitations affect MySQL's ability to do point-in-time recovery?

- 26.6.7: What is being done to correct the aforementioned limitations?

- 26.6.8: Do triggers work with replication?

- 26.6.9: How are actions carried out through triggers on a master replicated to a slave?

Questions and Answers

26.6.1: Do MySQL 5.0 stored procedures and functions work with replication?

Yes, standard actions carried out in stored procedures and functions are replicated from a master MySQL server to a slave server. There are a few limitations that are described in detail in Section 17.4, “Binary Logging of Stored Routines and Triggers”.

26.6.2: Are stored procedures and functions created on a master server replicated to a slave?

Yes, creation of stored procedures and functions carried out through normal DDL statements on a master server are replicated to a slave, so the objects will exist on both servers. `ALTER` and `DROP` statements for stored procedures and functions are also replicated.

26.6.3: How are actions that take place inside stored procedures and functions replicated?

MySQL records each DML event that occurs in a stored procedure and replicates those individual actions to a slave server. The actual calls made to execute stored procedures are not replicated.

Stored functions that change data are logged as function invocations, not as the DML events that occur inside each function.

26.6.4: Are there special security requirements for using stored procedures and functions together with replication?

Yes. Because a slave server has authority to execute any statement read from a master's binary log, special security constraints exist for using stored functions with replication. If replication or binary logging in general (for the purpose of point-in-time recovery) is active, then MySQL DBAs have two security options open to them:

1. Any user wishing to create stored functions must be granted the `SUPER` privilege.

2. Alternatively, a DBA can set the `log_bin_trust_function_creators` system variable to 1, which enables anyone with the standard `CREATE ROUTINE` privilege to create stored functions.

26.6.5: What limitations exist for replicating stored procedure and function actions?

Non-deterministic (random) or time-based actions embedded in stored procedures may not replicate properly. By their very nature, randomly produced results are not predictable and cannot be exactly reproduced, and therefore, random actions replicated to a slave will not mirror those performed on a master. Note that declaring stored functions to be `DETERMINISTIC` or setting the `log_bin_trust_function_creators` system variable to 0 will not allow random-valued operations to be invoked.

In addition, time-based actions cannot be reproduced on a slave because the timing of such actions in a stored procedure is not reproducible through the binary log used for replication. It records only DML events and does not factor in timing constraints.

Finally, non-transactional tables for which errors occur during large DML actions (such as bulk inserts) may experience replication issues in that a master may be partially updated from DML activity, but no updates are done to the slave because of the errors that occurred. A workaround is for a function's DML actions to be carried out with the `IGNORE` keyword so that updates on the master that cause errors are ignored and updates that do not cause errors are replicated to the slave.

26.6.6: Do the preceding limitations affect MySQL's ability to do point-in-time recovery?

The same limitations that affect replication do affect point-in-time recovery.

26.6.7: What is being done to correct the aforementioned limitations?

MySQL 5.1 implements row-based replication, which resolves the limitations mentioned earlier.

We do not plan to backport row-based replication to MySQL 5.0. For additional information, see Row-Based Replication, in the MySQL 5.1 Manual.

26.6.8: Do triggers work with replication?

Triggers and replication in MySQL 5.0 work in the same wasy as in most other database engines: Actions carried out through triggers on a master are not replicated to a slave server. Instead, triggers that exist on tables that reside on a MySQL master server need to be created on the corresponding tables on any MySQL slave servers so that the triggers activate on the slaves as well as the master.

26.6.9: How are actions carried out through triggers on a master replicated to a slave?

First, the triggers that exist on a master must be re-created on the slave server. Once this is done, the replication flow works as any other standard DML statement that participates in replication. For example, consider a table `EMP` that has an `AFTER` insert trigger, which exists on a master MySQL server. The same `EMP` table and `AFTER` insert trigger exist on the slave server as well. The replication flow would be:

1. An `INSERT` statement is made to `EMP`.

2. The `AFTER` trigger on `EMP` activates.

3. The `INSERT` statement is written to the binary log.

4. The replication slave picks up the `INSERT` statement to `EMP` and executes it.

5. The `AFTER` trigger on `EMP` that exists on the slave activates.

For answers to some general questions about MySQL stored procedures and stored functions, see Section A.4, “MySQL 5.0 FAQ — Stored Procedures”. Some common questions concerning MySQL triggers are adressed in Section A.5, “MySQL 5.0 FAQ — Triggers”.

## A.7. MySQL 5.0 FAQ — Views

Questions

- 26.7.1: Where can I find documentation for MySQL Views?

- 26.7.2: Is there a discussion forum for MySQL Views?

- 26.7.3: What happens to a view if an underlying table is dropped or renamed?

- 26.7.4: Does MySQL 5.0 have table snapshots?

- 26.7.5: Does MySQL 5.0 have materialized views?

- 26.7.6: Can you insert into views that are based on joins?

Questions and Answers

26.7.1: Where can I find documentation for MySQL Views?

See Chapter 19, Views.

26.7.2: Is there a discussion forum for MySQL Views?

Yes. See http://forums.mysql.com/list.php?100

26.7.3: What happens to a view if an underlying table is dropped or renamed?

After a view has been created, it is possible to drop or alter a table or view to which the definition refers. To check a view definition for problems of this kind, use the `CHECK TABLE` statement. (See Section 13.5.2.3, “`CHECK TABLE` Syntax”.)

26.7.4: Does MySQL 5.0 have table snapshots?

No.

26.7.5: Does MySQL 5.0 have materialized views?

No.

26.7.6: Can you insert into views that are based on joins?

It is possible, provided that your `INSERT` statement has a column list that makes it clear there's only one table involved.

You cannot insert into multiple tables with a single insert on a view.

## A.8. MySQL 5.0 FAQ — `INFORMATION_SCHEMA`

Questions

- 26.8.1: Where can I find documentation for the MySQL `INFORMATION_SCHEMA` database?

- 26.8.2: Is there a discussion forum for `INFORMATION_SCHEMA`?

- 26.8.3: Where can I find the ANSI SQL 2003 specification for `INFORMATION_SCHEMA`?

- 26.8.4: What is the difference between the Oracle Data Dictionary and MySQL's `INFORMATION_SCHEMA`?

- 26.8.5: Can I add to or otherwise modify the tables found in the `INFORMATION_SCHEMA` database?

Questions and Answers

26.8.1: Where can I find documentation for the MySQL `INFORMATION_SCHEMA` database?

See Chapter 20, The `INFORMATION_SCHEMA` Database

26.8.2: Is there a discussion forum for `INFORMATION_SCHEMA`?

See http://forums.mysql.com/list.php?101.

26.8.3: Where can I find the ANSI SQL 2003 specification for `INFORMATION_SCHEMA`?

Unfortunately, the official specifications are not freely available. (ANSI makes them available for purchase.) However, there are books available — such as SQL-99 Complete, Really by Peter Gulutzan and Trudy Pelzer — which give a comprehensive overview of the standard, including `INFORMATION_SCHEMA`.

26.8.4: What is the difference between the Oracle Data Dictionary and MySQL's `INFORMATION_SCHEMA`?

Both Oracle and MySQL provide metadata in tables. However, Oracle and MySQL use different table names and column names. MySQL's implementation is more similar to those found in DB2 and SQL Server, which also support `INFORMATION_SCHEMA` as defined in the SQL standard.

26.8.5: Can I add to or otherwise modify the tables found in the `INFORMATION_SCHEMA` database?

No. Since applications may rely on a certain standard structure, this should not be modified. For this reason, MySQL AB cannot support bugs or other issues which result from modifying `INFORMATION_SCHEMA` tables or data.

## A.9. MySQL 5.0 FAQ — Migration

Questions

- 26.9.1: Where can I find information on how to migrate from MySQL 4.1 to MySQL 5.0?

- 26.9.2: How has storage engine (table type) support changed in MySQL 5.0 from previous versions?

Questions and Answers

26.9.1: Where can I find information on how to migrate from MySQL 4.1 to MySQL 5.0?

For detailed upgrade information, see Section 2.4.16, “Upgrading MySQL”. We recommend that you do not skip a major version when upgrading, but rather complete the process in steps, upgrading from one major version to the next in each step. This may seem more complicated, but it will you save time and trouble — if you encounter problems during the upgrade, their origin will be easier to identify, either by you or — if you have a MySQL Network subscription — by MySQL support.

26.9.2: How has storage engine (table type) support changed in MySQL 5.0 from previous versions?

Storage engine support has changed as follows:

- Support for `ISAM` tables was removed in MySQL 5.0 and you should now use the `MyISAM` storage engine in place of `ISAM`. To convert a table `tblname` from `ISAM` to `MyISAM`, simply issue a statement such as this one:

      ALTER TABLE tblname ENGINE=MYISAM;

- Internal `RAID` for `MyISAM` tables was also removed in MySQL 5.0. This was formerly used to allow large tables in file systems that did not support file sizes greater than 2GB. All modern file systems allow for larger tables; in addition, there are now other solutions such as `MERGE` tables and views.

- The `VARCHAR` column type now retains trailing spaces in all storage engines.

- `MEMORY` tables (formerly known as `HEAP` tables) can also contain `VARCHAR` columns.

## A.10. MySQL 5.0 FAQ — Security

Questions

- 26.10.1: Where can I find documentation that addresses security issues for MySQL?

- 26.10.2: Does MySQL 5.0 have native support for SSL?

- 26.10.3: Is SSL support be built into MySQL binaries, or must I recompile the binary myself to enable it?

- 26.10.4: Does MySQL 5.0 have built-in authentication against LDAP directories?

- 26.10.5: Does MySQL 5.0 include support for Roles Based Access Control (RBAC)?

Questions and Answers

26.10.1: Where can I find documentation that addresses security issues for MySQL?

The best place to start is Section 5.7, “General Security Issues”.

Other portions of the MySQL Documentation which you may find useful with regard to specific security concerns include the following:

- Section 5.7.1, “General Security Guidelines”.

- Section 5.7.2, “Making MySQL Secure Against Attackers”.

- Section B.4.1, “How to Reset the Root Password”.

- Section 5.7.5, “How to Run MySQL as a Normal User”.

- Section 24.2.4.6, “User-Defined Function Security Precautions”.

- Section 5.7.3, “Security-Related mysqld Options”.

- Section 5.7.4, “Security Issues with `LOAD DATA LOCAL`”.

- Section 2.4.15, “Post-Installation Setup and Testing”.

- Section 2.4.18.1.11, “SELinux Notes”.

- Section 5.9.7.1, “Basic SSL Concepts”.

26.10.2: Does MySQL 5.0 have native support for SSL?

Most 5.0 binaries have support for SSL connections between the client and server. We can't currently build with the new YaSSL library everywhere, as it's still quite new and does not compile on all platforms yet. See Section 5.9.7, “Using Secure Connections”.

You can also tunnel a connection via SSH, if (for instance) if the client application doesn't support SSL connections. For an example, see Section 5.9.7.5, “Connecting to MySQL Remotely from Windows with SSH”.

26.10.3: Is SSL support be built into MySQL binaries, or must I recompile the binary myself to enable it?

Most 5.0 binaries have SSL enabled for client-server connections that are secured, authenticated, or both. However, the YaSSL library currently does not compile on all platforms. See Section 5.9.7, “Using Secure Connections”, for a complete listing of supported and unsupported platforms.

26.10.4: Does MySQL 5.0 have built-in authentication against LDAP directories?

No. Support for external authentication methods is on the MySQL roadmap as a “rolling feature”, which means that we plan to implement it in the future, but we have not yet determined when this will be done.

26.10.5: Does MySQL 5.0 include support for Roles Based Access Control (RBAC)?

No. Support for roles is on the MySQL roadmap as a “rolling feature”, which means that we plan to implement it in the future, but we have not yet determined when this will be done.

## A.11. MySQL 5.0 FAQ — MySQL Cluster

In the following section, we provide answers to questions that are most frequently asked about MySQL Cluster and the `NDB` storage engine.

Questions

- 26.11.1: What does “NDB” mean?

- 26.11.2: What's the difference in using Cluster vs using replication?

- 26.11.3: Do I need to do any special networking to run Cluster? How do computers in a cluster communicate?

- 26.11.4: How many computers do I need to run a cluster, and why?

- 26.11.5: What do the different computers do in a MySQL Cluster?

- 26.11.6: With which operating systems can I use Cluster?

- 26.11.7: What are the hardware requirements for running MySQL Cluster?

- 26.11.8: How much RAM do I need? Is it possible to use disk memory at all?

- 26.11.9: I'm trying to populate a Cluster database. The loading process terminates prematurely and I get an error message like this one:

      ERROR 1114: The table 'my_cluster_table' is full

  Why is this happening?

- 26.11.10: MySQL Cluster uses TCP/IP. Does this mean that I can run it over the Internet, with one or more nodes in remote locations?

- 26.11.11: Do I have to learn a new programming or query language to use MySQL Cluster?

- 26.11.12: How do I find out what an error or warning message means when using MySQL Cluster?

- 26.11.13: Is MySQL Cluster transaction-safe? What isolation levels are supported?

- 26.11.14: What storage engines are supported by MySQL Cluster?

- 26.11.15: Which versions of the MySQL software support Cluster? Do I have to compile from source?

- 26.11.16: In the event of a catastrophic failure — say, for instance, the whole city loses power and my UPS fails — would I lose all my data?

- 26.11.17: Is it possible to use `FULLTEXT` indexes with Cluster?

- 26.11.18: Can I run multiple nodes on a single computer?

- 26.11.19: Can I add nodes to a cluster without restarting it?

- 26.11.20: Are there any limitations that I should be aware of when using MySQL Cluster?

- 26.11.21: How do I import an existing MySQL database into a cluster?

- 26.11.22: How do cluster nodes communicate with one another?

- 26.11.23: What is an arbitrator?

- 26.11.24: What data types are supported by MySQL Cluster?

- 26.11.25: How do I start and stop MySQL Cluster?

- 26.11.26: What happens to cluster data when the cluster is shut down?

- 26.11.27: Is it helpful to have more than one management node for a cluster?

- 26.11.28: Can I mix different kinds of hardware and operating systems in one MySQL Cluster?

- 26.11.29: Can I run two data nodes on a single host? Two SQL nodes?

- 26.11.30: Can I use hostnames with MySQL Cluster?

- 26.11.31: How do I handle MySQL users in a Cluster having multiple MySQL servers?

Questions and Answers

26.11.1: What does “NDB” mean?

This stands for “Network Database”. `NDB` (also known as `NDB Cluster` or `NDBCLUSTER`) is the storage engine that enables clustering in MySQL.

26.11.2: What's the difference in using Cluster vs using replication?

In a replication setup, a master MySQL server updates one or more slaves. Transactions are committed sequentially, and a slow transaction can cause the slave to lag behind the master. This means that if the master fails, it is possible that the slave might not have recorded the last few transactions. If a transaction-safe engine such as `InnoDB` is being used, a transaction will either be complete on the slave or not applied at all, but replication does not guarantee that all data on the master and the slave will be consistent at all times. In MySQL Cluster, all data nodes are kept in synchrony, and a transaction committed by any one data node is committed for all data nodes. In the event of a data node failure, all remaining data nodes remain in a consistent state.

In short, whereas standard MySQL replication is asynchronous, MySQL Cluster is synchronous.

We have implemented (asynchronous) replication for Cluster in MySQL 5.1. This includes the capability to replicate both between two clusters, and from a MySQL cluster to a non-Cluster MySQL server. However, we do not plan to backport this functionality to MySQL 5.0.

26.11.3: Do I need to do any special networking to run Cluster? How do computers in a cluster communicate?

MySQL Cluster is intended to be used in a high-bandwidth environment, with computers connecting via TCP/IP. Its performance depends directly upon the connection speed between the cluster's computers. The minimum connectivity requirements for Cluster include a typical 100-megabit Ethernet network or the equivalent. We recommend you use gigabit Ethernet whenever available.

The faster SCI protocol is also supported, but requires special hardware. See Section 15.10, “Using High-Speed Interconnects with MySQL Cluster”, for more information about SCI.

26.11.4: How many computers do I need to run a cluster, and why?

A minimum of three computers is required to run a viable cluster. However, the minimum recommended number of computers in a MySQL Cluster is four: one each to run the management and SQL nodes, and two computers to serve as data nodes. The purpose of the two data nodes is to provide redundancy; the management node must run on a separate machine to guarantee continued arbitration services in the event that one of the data nodes fails.

To provide increased throughput and high availability, you should use multiple SQL nodes (MySQL Servers connected to the cluster). It is also possible (although not strictly necessary) to run multiple management servers.

26.11.5: What do the different computers do in a MySQL Cluster?

A MySQL Cluster has both a physical and logical organization, with computers being the physical elements. The logical or functional elements of a cluster are referred to as nodes, and a computer housing a cluster node is sometimes referred to as a cluster host. There are three types of nodes, each corresponding to a specific role within the cluster. These are:

- Management node (MGM node): Provides management services for the cluster as a whole, including startup, shutdown, backups, and configuration data for the other nodes. The management node server is implemented as the application ndb_mgmd; the management client used to control MySQL Cluster via the MGM node is ndb_mgm.

- Data node: Stores and replicates data. Data node functionality is handled by an instance of the NDB data node process ndbd.

- SQL node: This is simply an instance of MySQL Server (mysqld) that is built with support for the `NDB Cluster` storage engine and started with the --ndb-cluster option to enable the engine.

26.11.6: With which operating systems can I use Cluster?

MySQL Cluster is supported on most Unix-like operating systems, including Linux, Mac OS X, Solaris, BSD, HP-UX, AIX, and IRIX, among others, as well as Novell Netware. Cluster is not supported for Windows at this time. However, we are working to add Cluster support for other platforms, including Windows, and our goal is to offer MySQL Cluster on all platforms for which MySQL itself is supported.

For more detailed information concerning the level of support which is offered for MySQL Cluster on various operating system versions, OS distributions, and hardware platforms, please refer to http://www.mysql.com/support/supportedplatforms.html.

26.11.7: What are the hardware requirements for running MySQL Cluster?

Cluster should run on any platform for which NDB-enabled binaries are available. Naturally, faster CPUs and more memory will improve performance, and 64-bit CPUs will likely be more effective than 32-bit processors. There must be sufficient memory on machines used for data nodes to hold each node's share of the database (see How much RAM do I Need? for more information). Nodes can communicate via a standard TCP/IP network and hardware. For SCI support, special networking hardware is required.

26.11.8: How much RAM do I need? Is it possible to use disk memory at all?

In MySQL-5.0, Cluster is in-memory only. This means that all table data (including indexes) is stored in RAM. Therefore, if your data takes up 1GB of space and you want to replicate it once in the cluster, you need 2GB of memory to do so (1 GB per replica). This is in addition to the memory required by the operating system and any applications running on the cluster computers.

If a data node's memory usage exceeds what is available in RAM, then the system will attempt to use swap space up to the limit set for `DataMemory`. However, this will at best result in severely degraded performance, and may cause the node to be dropped due to slow response time (missed heartbeats). We do not recommend on relying on disk swapping in a production environment for this reason. In any case, once the `DataMemory` limit is reached, any operations requiring additional memory (such as inserts) will fail.

(We have implemented disk data storage for MySQL Cluster in MySQL 5.1, but we have no plans to add this capability in MySQL 5.0.)

You can use the following formula for obtaining a rough estimate of how much RAM is needed for each data node in the cluster:

    (SizeofDatabase × NumberOfReplicas × 1.1 ) / NumberOfDataNodes

To calculate the memory requirements more exactly requires determining, for each table in the cluster database, the storage space required per row (see Section 11.5, “Data Type Storage Requirements”, for details), and multiplying this by the number of rows. You must also remember to account for any column indexes as follows:

- Each primary key or hash index created for an `NDBCluster` table requires 21–25 bytes per record. These indexes use `IndexMemory`.

- Each ordered index requires 10 bytes storage per record, using `DataMemory`.

- Creating a primary key or unique index also creates an ordered index, unless this index is created with `USING HASH`. In other words:

  - A primary key or unique index on a Cluster table normally takes up 31 to 35 bytes per record.

  - However, if the primary key or unique index is created with `USING HASH`, then it requires only 21 to 25 bytes per record.

Note that creating MySQL Cluster tables with `USING HASH` for all primary keys and unique indexes will generally cause table updates to run more quickly — in some cases by a much as 20 to 30 percent faster than updates on tables where `USING HASH` was not used in creating primary and unique keys. This is due to the fact that less memory is required (because no ordered indexes are created), and that less CPU must be utilized (because fewer indexes must be read and possibly updated). However, it also means that queries that could otherwise use range scans must be satisfied by other means, which can result in slower selects.

When calculating Cluster memory requirements, you may find useful the `ndb_size.pl` utility which is available in recent MySQL 5.0 releases. This Perl script connects to a current (non-Cluster) MySQL database and creates a report on how much space that database would require if it used the `NDBCluster` storage engine. For more information, see Section 15.9.13, “ndb_size.pl”.

It is especially important to keep in mind that every MySQL Cluster table must have a primary key. The `NDB` storage engine creates a primary key automatically if none is defined, and this primary key is created without `USING HASH`.

There is no easy way to determine exactly how much memory is being used for storage of Cluster indexes at any given time; however, warnings are written to the Cluster log when 80% of available `DataMemory` or `IndexMemory` is in use, and again when use reaches 85%, 90%, and so on.

26.11.9: I'm trying to populate a Cluster database. The loading process terminates prematurely and I get an error message like this one:

    ERROR 1114: The table 'my_cluster_table' is full

Why is this happening?

The cause is very likely to be that your setup does not provide sufficient RAM for all table data and all indexes, including the primary key required by the `NDB` storage engine and automatically created in the event that the table definition does not include the definition of a primary key.

It is also worth noting that all data nodes should have the same amount of RAM, since no data node in a cluster can use more memory than the least amount available to any individual data node. In other words, if there are four computers hosting Cluster data nodes, and three of these have 3GB of RAM available to store Cluster data while the remaining data node has only 1GB RAM, then each data node can devote only 1GB to clustering.

26.11.10: MySQL Cluster uses TCP/IP. Does this mean that I can run it over the Internet, with one or more nodes in remote locations?

It is very unlikely that a cluster would perform reliably under such conditions, as MySQL Cluster was designed and implemented with the assumption that it would be run under conditions guaranteeing dedicated high-speed connectivity such as that found in a LAN setting using 100 Mbps or gigabit Ethernet — preferably the latter. We neither test nor warrant its performance using anything slower than this.

Also, it is extremely important to keep in mind that communications between the nodes in a MySQL Cluster are not secure; they are neither encrypted nor safeguarded by any other protective mechanism. The most secure configuration for a cluster is in a private network behind a firewall, with no direct access to any Cluster data or management nodes from outside. (For SQL nodes, you should take the same precautions as you would with any other instance of the MySQL server.)

26.11.11: Do I have to learn a new programming or query language to use MySQL Cluster?

No. Although some specialized commands are used to manage and configure the cluster itself, only standard (My)SQL queries and commands are required for the following operations:

- Creating, altering, and dropping tables

- Inserting, updating, and deleting table data

- Creating, changing, and dropping primary and unique indexes

Some specialized configuration parameters and files are required to set up a MySQL Cluster — see Section 15.4.4, “Configuration File”, for information about these.

A few simple commands are used in the MySQL Cluster management client for tasks such as starting and stopping cluster nodes. See Section 15.7.2, “Commands in the Management Client”.

26.11.12: How do I find out what an error or warning message means when using MySQL Cluster?

There are two ways in which this can be done:

- From within the mysql client, use SHOW ERRORS or SHOW WARNINGS immediately upon being notified of the error or warning condition. Errors and warnings also be displayed in MySQL Query Browser.

- From a system shell prompt, use perror --ndb `error_code`.

26.11.13: Is MySQL Cluster transaction-safe? What isolation levels are supported?

Yes: For tables created with the `NDB` storage engine, transactions are supported. In MySQL 5.0, Cluster supports only the `READ COMMITTED` transaction isolation level.

26.11.14: What storage engines are supported by MySQL Cluster?

Clustering in MySQL is supported only by the `NDB` storage engine. That is, in order for a table to be shared between nodes in a cluster, it must be created using `ENGINE=NDB` (or `ENGINE=NDBCLUSTER`, which is equivalent).

It is possible to create tables using other storage engines (such as `MyISAM` or `InnoDB`) on a MySQL server being used for clustering, but these non-`NDB` tables will not participate in the cluster; they are local to the individual MySQL server instance on which they are created.

26.11.15: Which versions of the MySQL software support Cluster? Do I have to compile from source?

MySQL Cluster is supported in all MySQL-max binaries in the 5.0 release series, except as noted in the following paragraph. You can determine whether your server has `NDB` support using either the `SHOW VARIABLES LIKE 'have_%'` or `SHOW ENGINES` statement. (See Section 5.3, “The mysqld-max Extended MySQL Server”, for more information.)

Linux users, please note that `NDB` is not included in the standard MySQL server RPMs. Beginning with MySQL 5.0.4, there are separate RPM packages for the NDB storage engine and accompanying management and other tools; see the NDB RPM Downloads section of the MySQL 5.0 Downloads page for these. (Prior to 5.0.4, you had to use the `-max` binaries supplied as `.tar.gz` archives. This is still possible, but is not required, so you can use your Linux distribution's RPM manager if you prefer.) You can also obtain NDB support by compiling the `-max` binaries from source, but it is not necessary to do so simply to use MySQL Cluster. To download the latest binary, RPM, or source distribution for MySQL 5.0, visit http://dev.mysql.com/downloads/mysql/5.0.html.

26.11.16: In the event of a catastrophic failure — say, for instance, the whole city loses power and my UPS fails — would I lose all my data?

All committed transactions are logged. Therefore, although it is possible that some data could be lost in the event of a catastrophe, this should be quite limited. Data loss can be further reduced by minimizing the number of operations per transaction. (It is not a good idea to perform large numbers of operations per transaction in any case.)

26.11.17: Is it possible to use `FULLTEXT` indexes with Cluster?

`FULLTEXT` indexing is not supported by the `NDB` storage engine in MySQL 5.0, or by any storage engine other than `MyISAM`. We are working to add this capability in a future release.

26.11.18: Can I run multiple nodes on a single computer?

It is possible but not advisable. One of the chief reasons to run a cluster is to provide redundancy. To enjoy the full benefits of this redundancy, each node should reside on a separate machine. If you place multiple nodes on a single machine and that machine fails, you lose all of those nodes. Given that MySQL Cluster can be run on commodity hardware loaded with a low-cost (or even no-cost) operating system, the expense of an extra machine or two is well worth it to safeguard mission-critical data. It also worth noting that the requirements for a cluster host running a management node are minimal. This task can be accomplished with a 200 MHz Pentium CPU and sufficient RAM for the operating system plus a small amount of overhead for the ndb_mgmd and ndb_mgm processes.

It is acceptable to run multiple cluster data nodes on a single host for learning about MySQL Cluster, or for testing purposes; however, this is not supported for production use.

26.11.19: Can I add nodes to a cluster without restarting it?

Not at present. A simple restart is all that is required for adding new MGM or SQL nodes to a Cluster. When adding data nodes the process is more complex, and requires the following steps:

1. Make a complete backup of all Cluster data.

2. Completely shut down the cluster and all cluster node processes.

3. Restart the cluster, using the `--initial` startup option.

4. Restore all cluster data from the backup.

In a future MySQL Cluster release series, we hope to implement a “hot” reconfiguration capability for MySQL Cluster to minimize (if not eliminate) the requirement for restarting the cluster when adding new nodes. However, this is not planned for MySQL 5.0.

26.11.20: Are there any limitations that I should be aware of when using MySQL Cluster?

Limitations on `NDB` tables in MySQL 5.0 include:

- Temporary tables are not supported; a `CREATE TEMPORARY TABLE` statement using `ENGINE=NDB` or `ENGINE=NDBCLUSTER` fails with an error.

- `FULLTEXT` indexes and index prefixes are not supported. Only complete columns may be indexed.

- Spatial data types are not supported. See Chapter 16, Spatial Extensions.

- Only complete rollbacks for transactions are supported. Partial rollbacks and rollbacks to save points are not supported.

- The maximum number of attributes allowed per table is 128, and attribute names cannot be any longer than 31 characters. For each table, the maximum combined length of the table and database names is 122 characters.

- The maximum size for a table row is 8 kilobytes, not counting `BLOB` values. There is no set limit for the number of rows per table. Table size limits depend on a number of factors, in particular on the amount of RAM available to each data node.

- The `NDB` engine does not support foreign key constraints. As with `MyISAM` tables, these are ignored.

For a complete listing of limitations in MySQL Cluster, see Section 15.11, “Known Limitations of MySQL Cluster”.

26.11.21: How do I import an existing MySQL database into a cluster?

You can import databases into MySQL Cluster much as you would with any other version of MySQL. Other than the limitations mentioned elsewhere in this FAQ and in Section 15.11, “Known Limitations of MySQL Cluster”, the only other special requirement is that any tables to be included in the cluster must use the `NDB` storage engine. This means that the tables must be created with `ENGINE=NDB` or `ENGINE=NDBCLUSTER`.

It is also possible to convert existing tables using other storage engines to `NDB Cluster` using one or more `ALTER TABLE` statement, but this requires an additional workaround. See Section 15.11, “Known Limitations of MySQL Cluster”, for details.

26.11.22: How do cluster nodes communicate with one another?

Cluster nodes can communicate via any of three different protocols: TCP/IP, SHM (shared memory), and SCI (Scalable Coherent Interface). Where available, SHM is used by default between nodes residing on the same cluster host; however, this is considered experimental in MySQL 5.0. SCI is a high-speed (1 gigabit per second and higher), high-availability protocol used in building scalable multi-processor systems; it requires special hardware and drivers. See Section 15.10, “Using High-Speed Interconnects with MySQL Cluster”, for more about using SCI as a transport mechanism in MySQL Cluster.

26.11.23: What is an arbitrator?

If one or more nodes in a cluster fail, it is possible that not all cluster nodes will be able to “see” one another. In fact, it is possible that two sets of nodes might become isolated from one another in a network partitioning, also known as a “split brain” scenario. This type of situation is undesirable because each set of nodes tries to behave as though it is the entire cluster.

When cluster nodes go down, there are two possibilities. If more than 50% of the remaining nodes can communicate with each other, we have what is sometimes called a “majority rules” situation, and this set of nodes is considered to be the cluster. The arbitrator comes into play when there is an even number of nodes: in such cases, the set of nodes to which the arbitrator belongs is considered to be the cluster, and nodes not belonging to this set are shut down.

The preceding information is somewhat simplified. A more complete explanation taking into account node groups follows:

When all nodes in at least one node group are alive, network partitioning is not an issue, because no one portion of the cluster can form a functional cluster. The real problem arises when no single node group has all its nodes alive, in which case network partitioning (the “split-brain” scenario) becomes possible. Then an arbitrator is required. All cluster nodes recognize the same node as the arbitrator, which is normally the management server; however, it is possible to configure any of the MySQL Servers in the cluster to act as the arbitrator instead. The arbitrator accepts the first set of cluster nodes to contact it, and tells the remaining set to shut down. Arbitrator selection is controlled by the `ArbitrationRank` configuration parameter for MySQL Server and management server nodes. (See Section 15.4.4.4, “Defining the Management Server”, for details.) It should also be noted that the role of arbitrator does not in and of itself impose any heavy demands upon the host so designated, and thus the arbitrator host does not need to be particularly fast or to have extra memory especially for this purpose.

26.11.24: What data types are supported by MySQL Cluster?

MySQL Cluster supports all of the usual MySQL data types, with the exception of those associated with MySQL's spatial extensions. (See Chapter 16, Spatial Extensions.) In addition, there are some differences with regard to indexes when used with `NDB` tables. Note: MySQL Cluster tables (that is, tables created with `ENGINE=NDBCLUSTER`) have only fixed-width rows. This means that (for example) each record containing a `VARCHAR(255)` column will require space for 255 characters (as required for the character set and collation being used for the table), regardless of the actual number of characters stored therein. This issue is expected to be fixed in a future MySQL release series.

See Section 15.11, “Known Limitations of MySQL Cluster”, for more information about these issues.

26.11.25: How do I start and stop MySQL Cluster?

It is necessary to start each node in the cluster separately, in the following order:

1. Start the management node with the ndb_mgmd command.

2. Start each data node with the ndbd command.

3. Start each MySQL server (SQL node) using mysqld_safe --user=mysql &.

Each of these commands must be run from a system shell on the machine housing the affected node. (You do not have to be physically present at the machine — a remote login shell can be used for this purpose.) You can verify that the cluster is running by starting the MGM management client ndb_mgm on the machine housing the MGM node and issuing the `SHOW` or `ALL STATUS` command.

To shut down a running cluster, issue the command `SHUTDOWN` in the MGM client. Alternatively, you may enter the following command in a system shell on the machine hosting the MGM node:

    shell> ndb_mgm -e "SHUTDOWN"

(Note that the quotation marks are optional here; the `SHUTDOWN` command itself is not case-sensitive.)

Either of these commands causes the ndb_mgm, ndb_mgm, and any ndbd processes to terminate gracefully. MySQL servers running as Cluster SQL nodes can be stopped using mysqladmin shutdown.

For more information, see Section 15.7.2, “Commands in the Management Client”, and Section 15.3.6, “Safe Shutdown and Restart”.

26.11.26: What happens to cluster data when the cluster is shut down?

The data that was held in memory by the cluster's data nodes is written to disk, and is reloaded into memory the next time that the cluster is started.

26.11.27: Is it helpful to have more than one management node for a cluster?

It can be helpful as a fail-safe. Only one MGM node controls the cluster at any given time, but it is possible to configure one MGM as primary, and one or more additional management nodes to take over in the event that the primary MGM node fails.

See Section 15.4.4, “Configuration File”, for information on how to configure MySQL Cluster management nodes.

26.11.28: Can I mix different kinds of hardware and operating systems in one MySQL Cluster?

Yes, so long as all machines and operating systems have the same “endianness” (all big-endian or all little-endian). It is also possible to use different MySQL Cluster releases on different nodes. However, we recommend this be done only as part of a rolling upgrade procedure (see Section 15.5.1, “Performing a Rolling Restart of the Cluster”).

26.11.29: Can I run two data nodes on a single host? Two SQL nodes?

Yes, it is possible to do this. In the case of multiple data nodes, it is advisable (but not required) for each node to use a different data directory. If you want to run multiple SQL nodes on one machine, each instance of mysqld must use a different TCP/IP port. However, running more than one cluster node of a given type per machine is not supported for production use.

26.11.30: Can I use hostnames with MySQL Cluster?

Yes, it is possible to use DNS and DHCP for cluster hosts. However, if your application requires “five nines” availability, we recommend using fixed IP addresses. Making communication between Cluster hosts dependent on services such as DNS and DHCP introduces additional points of failure, and the fewer of these, the better.

26.11.31: How do I handle MySQL users in a Cluster having multiple MySQL servers?

MySQL user accounts and privileges are not automatically propagated between different MySQL servers accessing the same MySQL Cluster. Therefore, you must make sure that these are copied between the SQL nodes yourself.

## A.12. MySQL 5.0 FAQ — MySQL Chinese, Japanese, and Korean Character Sets

This set of Frequently Asked Questions derives from the experience of MySQL's Support and Development groups in handling many inquiries about CJK (Chinese-Japanese-Korean) issues.

Questions

- 26.12.1: I have inserted CJK characters into my table. Why does `SELECT` display them as “?” characters?

- 26.12.2: What GB (Chinese) character sets does MySQL support?

- 26.12.3: What problems should I be aware of when working with the Big5 Chinese character set?

- 26.12.4: Why do Japanese character set conversions fail?

- 26.12.5: What should I do if I want to convert SJIS `81CA` to `cp932`?

- 26.12.6: How does MySQL represent the Yen (`¥`) sign?

- 26.12.7: Do MySQL plan to make a separate character set where `5C` is the Yen sign, as at least one other major DBMS does?

- 26.12.8: Of what issues should I be aware when working with Korean character sets in MySQL?

- 26.12.9: Why do I get Data truncated error messages?

- 26.12.10: Why does my GUI front end or browser not display CJK characters correctly in my application using Access, PHP, or another API?

- 26.12.11: I've upgraded to MySQL 5.0. How can I revert to behavior like that in MySQL 4.0 with regard to character sets?

- 26.12.12: Why do some `LIKE` and `FULLTEXT` searches with CJK characters fail?

- 26.12.13: What CJK character sets are available in MySQL?

- 26.12.14: How do I know whether character `X` is available in all character sets?

- 26.12.15: Why don't CJK strings sort correctly in Unicode? (I)

- 26.12.16: Why don't CJK strings sort correctly in Unicode? (II)

- 26.12.17: Why are my supplementary characters rejected by MySQL?

- 26.12.18: Shouldn't it be “CJKV”?

- 26.12.19: Does MySQL allow CJK characters to be used in database and table names?

- 26.12.20: Where can I find translations of the MySQL Manual into Chinese, Japanese, and Korean?

- 26.12.21: Where can I get help with CJK and related issues in MySQL?

Questions and Answers

26.12.1: I have inserted CJK characters into my table. Why does `SELECT` display them as “?” characters?

This problem is usually due to a setting in MySQL that doesn't match the settings for the application program or the operating system. Here are some common steps for correcting these types of issues:

- Be certain of what MySQL version you are using.

  Use the statement `SELECT VERSION();` to determine this.

- Make sure that the database is actually using the desired character set.

  People often think that the client character set is always the same as either the server character set or the character set used for display purposes. However, both of these are false assumptions. You can make sure by checking the result of `SHOW CREATE TABLE `tablename`` or — better — yet by using this statement:

      SELECT character_set_name, collation_name
          FROM information_schema.columns
          WHERE table_schema = your_database_name
              AND table_name = your_table_name
              AND column_name = your_column_name;

- Determine the hexadecimal value of the character or characters that are not being displayed correctly.

  You can obtain this information for a column `column_name` in the table `table_name` using the following query:

      SELECT HEX(column_name)
      FROM table_name;

  `3F` is the encoding for the `?` character; this means that `?` is the character actually stored in the column. This most often happens because of a problem converting a particular character from your client character set to the target character set.

- Make sure that a round trip possible — that is, when you select `literal` (or `_introducer hexadecimal-value`), you obtain `literal` as a result.

  For example, the Japanese Katakana character Pe (`ペ'`) exists in all CJK character sets, and has the code point value (hexadecimal coding) `0x30da`. To test a round trip for this character, use this query:

      SELECT 'ペ' AS `ペ`;         /* or SELECT _ucs2 0x30da; */

  If the result is not also `ペ`, then the round trip has failed.

  For bug reports regarding such failures, we might ask you to follow up with `SELECT HEX('ペ');`. Then we can determine whether the client encoding is correct.

- Make sure that the problem is not with the browser or other application, rather than with MySQL.

  Use the mysql client program (on Windows: mysql.exe) to accomplish this task. If mysql displays correctly but your application doesn't, then your problem is probably due to system settings.

  To find out what your settings are, use the `SHOW VARIABLES` statement, whose output should resemble what is shown here:

      mysql> SHOW VARIABLES LIKE 'char%';
      +--------------------------+----------------------------------------+
      | Variable_name            | Value                                  |
      +--------------------------+----------------------------------------+
      | character_set_client     | utf8                                   |
      | character_set_connection | utf8                                   |
      | character_set_database   | latin1                                 |
      | character_set_filesystem | binary                                 |
      | character_set_results    | utf8                                   |
      | character_set_server     | latin1                                 |
      | character_set_system     | utf8                                   |
      | character_sets_dir       | /usr/local/mysql/share/mysql/charsets/ |
      +--------------------------+----------------------------------------+
      8 rows in set (0.03 sec)

  These are typical character-set settings for an international-oriented client (notice the use of `utf8` Unicode) connected to a server in the West (`latin1` is a West Europe character set and a default for MySQL).

  Although Unicode (usually the `utf8` variant on Unix, and the `ucs2` variant on Windows) is preferable to Latin, it's often not what your operating system utilities support best. Many Windows users find that a Microsoft character set, such as `cp932` for Japanese Windows, is what's suitable.

  If you cannot control the server settings, and you have no idea what your underlying computer is, then try changing to a common character set for the country that you're in (`euckr` = Korea; `gb2312` or `gbk` = People's Republic of China; `big5` = Taiwan; `sjis`, `ujis`, `cp932`, or `eucjpms` = Japan; `ucs2` or `utf8` = anywhere). Usually it is necessary to change only the client and connection and results settings. There is a simple statement which changes all three at once: `SET NAMES`. For example:

      SET NAMES 'big5';

  Once the setting is correct, you can make it permanent by editing `my.cnf` or `my.ini`. For example you might add lines looking like these:

      [mysqld]
      character-set-server=big5
      [client]
      default-character-set=big5

  It is also possible that there are issues with the API configuration setting being used in your application; see Why does my GUI front end or browser not display CJK characters correctly...? for more information.

26.12.2: What GB (Chinese) character sets does MySQL support?

MySQL supports the two common variants of the GB (Guojia Biaozhun, or National Standard) character sets which are official in the People's Republic of China: `gb2312` and `gbk`. Sometimes people try to insert `gbk` characters into `gb2312`, and it works most of the time because `gbk` is a superset of `gb2312` — but eventually they try to insert a rarer Chinese character and it doesn't work. (See Bug#16072 for an example).

Here, we try to clarify exactly what characters are legitimate in `gb2312` or `gbk`, with reference to the official documents. Please check these references before reporting `gb2312` or `gbk` bugs.

- For a complete listing of the `gb2312` characters, ordered according to the `gb2312_chinese_ci` collation: http://d.udm.net/bar/~bar/charts/gb2312_chinese_ci.html.

- MySQL's `gbk` is in reality “Microsoft code page 936”. This differs from the official `gbk` for characters `A1A4` (middle dot), `A1AA` (em dash), `A6E0-A6F5`, and `A8BB-A8C0`. For a listing of the differences, see http://recode.progiciels-bpi.ca/showfile.html?name=dist/libiconv/gbk.h.

- For a listing of `gbk`/Unicode mappings, see http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP936.TXT.

- For MySQL's listing of `gbk` characters, see http://d.udm.net/bar/~bar/charts/gbk_chinese_ci.html.

26.12.3: What problems should I be aware of when working with the Big5 Chinese character set?

MySQL supports the Big5 character set which is common in Hong Kong and Taiwan (Republic of China). MySQL's `big5` is in reality Microsoft code page 950, which is very similar to the original `big5` character set. We changed to this character set starting with MySQL version 4.1.16 / 5.0.16 (as a result of Bug#12476). For example, the following statements work in current versions of MySQL, but not in old versions:

    mysql> CREATE TABLE big5 (BIG5 CHAR(1) CHARACTER SET BIG5);
    Query OK, 0 rows affected (0.13 sec)

    mysql> INSERT INTO big5 VALUES (0xf9dc);
    Query OK, 1 row affected (0.00 sec)

    mysql> SELECT * FROM big5;
    +------+
    | big5 |
    +------+
    | 嫺  |
    +------+
    1 row in set (0.02 sec)

A feature request for adding `HKSCS` extensions has been filed. People who need this extension may find the suggested patch for Bug#13577 to be of interest.

26.12.4: Why do Japanese character set conversions fail?

MySQL supports the `sjis`, `ujis`, `cp932`, and `eucjpms` character sets, as well as Unicode. A common need is to convert between character sets. For example, there might be a Unix server (typically with `sjis` or `ujis`) and a Windows client (typically with `cp932`).

In the following conversion table, the `ucs2` column represents the source, and the `sjis`, `cp932`, `ujis`, and `eucjpms` columns represent the destinations — that is, the last 4 columns provide the hexadecimal result when we use `CONVERT(ucs2)` or we assign a `ucs2` column containing the value to an `sjis`, `cp932`, `ujis`, or `eucjpms` column.

Character Name | ucs2 | sjis | cp932 | ujis | eucjpms
--- | --- | --- | --- | --- | ---
BROKEN BAR | 00A6 | 3F | 3F | 8FA2C3 | 3F
FULLWIDTH BROKEN BAR | FFE4 | 3F | FA55 | 3F | 8FA2
YEN SIGN | 00A5 | 3F | 3F | 20 | 3F
FULLWIDTH YEN SIGN | FFE5 | 818F | 818F | A1EF | 3F
TILDE | 007E | 7E | 7E | 7E | 7E
OVERLINE | 203E | 3F | 3F | 20 | 3F
HORIZONTAL BAR | 2015 | 815C | 815C | A1BD | A1BD
EM DASH | 2014 | 3F | 3F | 3F | 3F
REVERSE SOLIDUS | 005C | 815F | 5C | 5C | 5C
FULLWIDTH "" | FF3C | 3F | 815F | 3F | A1C0
WAVE DASH | 301C | 8160 | 3F | A1C1 | 3F
FULLWIDTH TILDE | FF5E | 3F | 8160 | 3F | A1C1
DOUBLE VERTICAL LINE | 2016 | 8161 | 3F | A1C2 | 3F
PARALLEL TO | 2225 | 3F | 8161 | 3F | A1C2
MINUS SIGN | 2212 | 817C | 3F | A1DD | 3F
FULLWIDTH HYPHEN-MINUS | FF0D | 3F | 817C | 3F | A1DD
CENT SIGN | 00A2 | 8191 | 3F | A1F1 | 3F
FULLWIDTH CENT SIGN | FFE0 | 3F | 8191 | 3F | A1F1
POUND SIGN | 00A3 | 8192 | 3F | A1F2 | 3F
FULLWIDTH POUND SIGN | FFE1 | 3F | 8192 | 3F | A1F2
NOT SIGN | 00AC | 81CA | 3F | A2CC | 3F
FULLWIDTH NOT SIGN | FFE2 | 3F | 81CA | 3F | A2CC

Now consider this portion of the table:

| ucs2 | sjis | cp932
--- | --- | --- | ---
NOT SIGN | 00AC | 81CA | 3F
FULLWIDTH NOT SIGN | FFE2 | 3F | 81CA

This means that MySQL converts the `NOT SIGN` (Unicode `U+00AC`) to `sjis` code point `0x81CA` and to `cp932` code point `3F`. (`3F` is the question mark (“?”) — this is what is always used when the conversion cannot be performed.

26.12.5: What should I do if I want to convert SJIS `81CA` to `cp932`?

Our answer is: “?”. There are serious complaints about this: many people would prefer a “loose” conversion, so that `81CA (NOT SIGN)` in `sjis` becomes `81CA (FULLWIDTH NOT SIGN)` in `cp932`. We are considering a change to this behavior.

26.12.6: How does MySQL represent the Yen (`¥`) sign?

A problem arises because some versions of Japanese character sets (both `sjis` and `euc`) treat `5C` as a reverse solidus (`\` — also known as a backslash), and others treat it as a yen sign (`¥`).

MySQL follows only one version of the JIS (Japanese Industrial Standards) standard description. In MySQL, `5C` is always the reverse solidus (`\`).

26.12.7: Do MySQL plan to make a separate character set where `5C` is the Yen sign, as at least one other major DBMS does?

This is one possible solution to the Yen sign issue; however, this will not happen in MySQL 5.1 or 5.2.

26.12.8: Of what issues should I be aware when working with Korean character sets in MySQL?

In theory, while there have been several versions of the `euckr` (Extended Unix Code Korea) character set, only one problem has been noted.

We use the “ASCII” variant of EUC-KR, in which the code point `0x5c` is REVERSE SOLIDUS, that is `\`, instead of the “KS-Roman” variant of EUC-KR, in which the code point `0x5c` is `WON SIGN`(`₩`). This means that you cannot convert Unicode `U+20A9` to `euckr`:

    mysql> SELECT
        ->     CONVERT('₩' USING euckr) AS euckr,
        ->     HEX(CONVERT('₩' USING euckr)) AS hexeuckr;
    +-------+----------+
    | euckr | hexeuckr |
    +-------+----------+
    | ?     | 3F       |
    +-------+----------+
    1 row in set (0.00 sec)

MySQL's graphic Korean chart is here: http://d.udm.net/bar/~bar/charts/euckr_korean_ci.html.

26.12.9: Why do I get Data truncated error messages?

For illustration, we'll create a table with one Unicode (`ucs2`) column and one Chinese (`gb2312`) column.

    mysql> CREATE TABLE ch
        -> (ucs2 CHAR(3) CHARACTER SET ucs2,
        -> gb2312 CHAR(3) CHARACTER SET gb2312);
    Query OK, 0 rows affected (0.05 sec)

We'll try to place the rare character `汌` in both columns.

    mysql> INSERT INTO ch VALUES ('A汌B','A汌B');
    Query OK, 1 row affected, 1 warning (0.00 sec)

Ah, there's a warning. Let's see what it is.

    mysql> SHOW WARNINGS;
    +---------+------+---------------------------------------------+
    | Level   | Code | Message                                     |
    +---------+------+---------------------------------------------+
    | Warning | 1265 | Data truncated for column 'gb2312' at row 1 |
    +---------+------+---------------------------------------------+
    1 row in set (0.00 sec)

So it's a warning about the `gb2312` column only.

    mysql> SELECT ucs2,HEX(ucs2),gb2312,HEX(gb2312) FROM ch;
    +-------+--------------+--------+-------------+
    | ucs2  | HEX(ucs2)    | gb2312 | HEX(gb2312) |
    +-------+--------------+--------+-------------+
    | A汌B | 00416C4C0042 | A?B    | 413F42      |
    +-------+--------------+--------+-------------+
    1 row in set (0.00 sec)

There are several things that need explanation here.

1. The fact that it's a “warning” rather than an “error” is characteristic of MySQL. We like to try to do what we can, to get the best fit, rather than give up.

2. The `汌` character isn't in the `gb2312` character set. We described that problem earlier.

3. Admittedly the message is misleading. We didn't “truncate” in this case, we replaced with a question mark. We've had a complaint about this message (See Bug#9337). But until we come up with something better, just accept that error/warning code 2165 can mean a variety of things.

4. With `SQL_MODE=TRADITIONAL`, there would be an error message, but instead of error 2165 you would see: `ERROR 1406 (22001): Data too long for column 'gb2312' at row 1`.

26.12.10: Why does my GUI front end or browser not display CJK characters correctly in my application using Access, PHP, or another API?

Obtain a direct connection to the server using the mysql client (Windows: mysql.exe), and try the same query there. If mysql responds correctly, then the trouble may be that your application interface requires initialization. Use mysql to tell you what character set or sets it uses with the statement `SHOW VARIABLES LIKE 'char%';`. If you are using Access, then you are most likely connecting with MyODBC. In this case, you should check Section 23.1.3, “Connector/ODBC Configuration”. If, for instance, you use `big5`, you would enter `SET NAMES 'big5'`. (Note that no `;` is required in this case). If you are using ASP, you might need to add `SET NAMES` in the code. Here is an example that has worked in the past:

    <%
    Session.CodePage=0
    Dim strConnection
    Dim Conn
    strConnection="driver={MySQL ODBC 3.51 Driver};server=server;uid=username;" \
                   & "pwd=password;database=database;stmt=SET NAMES 'big5';"
    Set Conn = Server.CreateObject("ADODB.Connection")
    Conn.Open strConnection
    %>

In much the same way, if you are using any character set other than `latin1` with Connector/NET, then you must specify the character set in the connection string. See Section 23.2.5.1, “Connecting to MySQL Using Connector/NET”, for more information.

If you are using PHP, try this:

    <?php
      $link = mysql_connect($host, $usr, $pwd);

      mysql_select_db($db);

      if( mysql_error() ) { print "Database ERROR: " . mysql_error(); }
      mysql_query("SET NAMES 'utf8'", $link);
    ?>

In this case, we used `SET NAMES` to change `character_set_client` and `character_set_connection` and `character_set_results`.

We encourage the use of the newer `mysqli` extension, rather than `mysql`. Using `mysqli`, the previous example could be rewritten as shown here:

    <?php
      $link = new mysqli($host, $usr, $pwd, $db);

      if( mysqli_connect_errno() )
      {
        printf("Connect failed: %s\n", mysqli_connect_error());
        exit();
      }

      $link->query("SET NAMES 'utf8'");
    ?>

Another issue often encountered in PHP applications has to do with assumptions made by the browser. Sometimes adding or changing a `<meta>` tag suffices to correct the problem: for example, to insure that the user agent interprets page content as `UTF-8`, you should include `<meta http-equiv="Content-Type" content="text/html; charset=utf-8">` in the `<head>` of the HTML page.

If you are using Connector/J, see Section 23.4.4.4, “Using Character Sets and Unicode”.

26.12.11: I've upgraded to MySQL 5.0. How can I revert to behavior like that in MySQL 4.0 with regard to character sets?

In MySQL Version 4.0, there was a single “global” character set for both server and client, and the decision as to which character to use was made by the server administrator. This changed starting with MySQL Version 4.1. What happens now is a “handshake”, as described in Section 10.4, “Connection Character Sets and Collations”:

> When a client connects, it sends to the server the name of the character set that it wants to use. The server uses the name to set the `character_set_client`, `character_set_results`, and `character_set_connection` system variables. In effect, the server performs a `SET NAMES` operation using the character set name.

The effect of this is that you cannot control the client character set by starting mysqld with `--character-set-server=utf8`. However, some of our Asian customers have said that prefer the MySQL 4.0 behavior. To make it possible to retain this behavior, we added a mysqld switch, `--character-set-client-handshake`, which can be turned off with `--skip-character-set-client-handshake`. If you start mysqld with `--skip-character-set-client-handshake`, then, when a client connects, it sends to the server the name of the character set that it wants to use — however, the server ignores this request from the client.

By way of example, suppose that your favorite server character set is `latin1` (unlikely in a CJK area, but this is the default value). Suppose further that the client uses `utf8` because this is what the client's operating system supports. Now, start the server with `latin1` as its default character set:

    mysqld --character-set-server=latin1

And then start the client with the default character set `utf8`:

    mysql --default-character-set=utf8

The current settings can be seen by viewing the output of `SHOW VARIABLES`:

    mysql> SHOW VARIABLES LIKE 'char%';
    +--------------------------+----------------------------------------+
    | Variable_name            | Value                                  |
    +--------------------------+----------------------------------------+
    | character_set_client     | utf8                                   |
    | character_set_connection | utf8                                   |
    | character_set_database   | latin1                                 |
    | character_set_filesystem | binary                                 |
    | character_set_results    | utf8                                   |
    | character_set_server     | latin1                                 |
    | character_set_system     | utf8                                   |
    | character_sets_dir       | /usr/local/mysql/share/mysql/charsets/ |
    +--------------------------+----------------------------------------+
    8 rows in set (0.01 sec)

Now stop the client, and then stop the server using mysqladmin. Then start the server again, but this time tell it to skip the handshake like so:

    mysqld --character-set-server=utf8 --skip-character-set-client-handshake

Start the client with `utf8` once again as the default character set, then display the current settings:

    mysql> SHOW VARIABLES LIKE 'char%';
    +--------------------------+----------------------------------------+
    | Variable_name            | Value                                  |
    +--------------------------+----------------------------------------+
    | character_set_client     | latin1                                 |
    | character_set_connection | latin1                                 |
    | character_set_database   | latin1                                 |
    | character_set_filesystem | binary                                 |
    | character_set_results    | latin1                                 |
    | character_set_server     | latin1                                 |
    | character_set_system     | utf8                                   |
    | character_sets_dir       | /usr/local/mysql/share/mysql/charsets/ |
    +--------------------------+----------------------------------------+
    8 rows in set (0.01 sec)

As you can see by comparing the differing results from `SHOW VARIABLES`, the server ignores the client's initial settings if the `--skip-character-set-client-handshake` is used.

26.12.12: Why do some `LIKE` and `FULLTEXT` searches with CJK characters fail?

There is a very simple problem with `LIKE` searches on `BINARY` and `BLOB` columns: we need to know the end of a character. With multi-byte character sets, different characters might have different octet lengths. For example, in `utf8`, `A` requires one byte but `ペ` requires three bytes, as shown here:

    +-------------------------+---------------------------+
    | OCTET_LENGTH(_utf8 'A') | OCTET_LENGTH(_utf8 'ペ') |
    +-------------------------+---------------------------+
    |                       1 |                         3 |
    +-------------------------+---------------------------+
    1 row in set (0.00 sec)

If we don't know where the first character ends, then we don't know where the second character begins, in which case even very simple searches such as `LIKE '_A%'` fail. The solution is to use a regular CJK character set in the first place, or to convert to a CJK character set before comparing.

This is one reason why MySQL cannot allow encodings of nonexistent characters. If it is not strict about rejecting bad input, then it has no way of knowing where characters end.

For `FULLTEXT` searches, we need to know where words begin and end. With Western languages, this is rarely a problem because most (if not all) of these use an easy-to-identify word boundary — the space character. However, this is not usually the case with Asian writing. We could use arbitrary halfway measures, like assuming that all Han characters represent words, or (for Japanese) depending on changes from Katakana to Hiragana due to grammatical endings. However, the only sure solution requires a comprehensive word list, which means that we would have to include a dictionary in the server for each Asian language supported. This is simply not feasible.

26.12.13: What CJK character sets are available in MySQL?

The list of CJK character sets may vary depending on your MySQL version. For example, the `eucjpms` character set was not supported prior to MySQL 5.0.3 (see Section G.1.27, “Changes in release 5.0.3 (23 March 2005: Beta)”). However, since the name of the applicable language appears in the `DESCRIPTION` column for every entry in the `INFORMATION_SCHEMA.CHARACTER_SETS` table, you can obtain a current list of all the non-Unicode CJK character sets using this query:

    mysql> SELECT CHARACTER_SET_NAME, DESCRIPTION
        -> FROM INFORMATION_SCHEMA.CHARACTER_SETS
        -> WHERE DESCRIPTION LIKE '%Chinese%'
        -> OR DESCRIPTION LIKE '%Japanese%'
        -> OR DESCRIPTION LIKE '%Korean%'
        -> ORDER BY CHARACTER_SET_NAME;
    +--------------------+---------------------------+
    | CHARACTER_SET_NAME | DESCRIPTION               |
    +--------------------+---------------------------+
    | big5               | Big5 Traditional Chinese  |
    | cp932              | SJIS for Windows Japanese |
    | eucjpms            | UJIS for Windows Japanese |
    | euckr              | EUC-KR Korean             |
    | gb2312             | GB2312 Simplified Chinese |
    | gbk                | GBK Simplified Chinese    |
    | sjis               | Shift-JIS Japanese        |
    | ujis               | EUC-JP Japanese           |
    +--------------------+---------------------------+
    8 rows in set (0.01 sec)

(See Section 20.9, “The `INFORMATION_SCHEMA CHARACTER_SETS` Table”, for more information.)

26.12.14: How do I know whether character `X` is available in all character sets?

The majority of simplified Chinese and basic non-halfwidth Japanese Kana characters appear in all CJK character sets. This stored procedure accepts a `UCS-2` Unicode character, converts it to all other character sets, and displays the results in hexadecimal.

    DELIMITER //

    CREATE PROCEDURE p_convert(ucs2_char CHAR(1) CHARACTER SET ucs2)
    BEGIN

    CREATE TABLE tj
                 (ucs2 CHAR(1) character set ucs2,
                  utf8 CHAR(1) character set utf8,
                  big5 CHAR(1) character set big5,
                  cp932 CHAR(1) character set cp932,
                  eucjpms CHAR(1) character set eucjpms,
                  euckr CHAR(1) character set euckr,
                  gb2312 CHAR(1) character set gb2312,
                  gbk CHAR(1) character set gbk,
                  sjis CHAR(1) character set sjis,
                  ujis CHAR(1) character set ujis);

    INSERT INTO tj (ucs2) VALUES (ucs2_char);

    UPDATE tj SET utf8=ucs2,
                  big5=ucs2,
                  cp932=ucs2,
                  eucjpms=ucs2,
                  euckr=ucs2,
                  gb2312=ucs2,
                  gbk=ucs2,
                  sjis=ucs2,
                  ujis=ucs2;

    /* If there's a conversion problem, UPDATE will produce a warning. */

    SELECT hex(ucs2) AS ucs2,
           hex(utf8) AS utf8,
           hex(big5) AS big5,
           hex(cp932) AS cp932,
           hex(eucjpms) AS eucjpms,
           hex(euckr) AS euckr,
           hex(gb2312) AS gb2312,
           hex(gbk) AS gbk,
           hex(sjis) AS sjis,
           hex(ujis) AS ujis
    FROM tj;

    DROP TABLE tj;

    END//

The input can be any single `ucs2` character, or it can be the code point value (hexadecimal representation) of that character. For example, from Unicode's list of `ucs2` encodings and names (http://www.unicode.org/Public/UNIDATA/UnicodeData.txt), we know that the Katakana character Pe appears in all CJK character sets, and that its code point value is `0x30da`. If we use this value as the argument to `p_convert()`, the result is as shown here:

    mysql> CALL p_convert(0x30da)//
    +------+--------+------+-------+---------+-------+--------+------+------+------+
    | ucs2 | utf8   | big5 | cp932 | eucjpms | euckr | gb2312 | gbk  | sjis | ujis |
    +------+--------+------+-------+---------+-------+--------+------+------+------+
    | 30DA | E3839A | C772 | 8379  | A5DA    | ABDA  | A5DA   | A5DA | 8379 | A5DA |
    +------+--------+------+-------+---------+-------+--------+------+------+------+
    1 row in set (0.04 sec)

Since none of the column values is `3F` — that is, the question mark character (`?`) — we know that every conversion worked.

26.12.15: Why don't CJK strings sort correctly in Unicode? (I)

Sometimes people observe that the result of a `utf8_unicode_ci` or `ucs2_unicode_ci` search, or of an `ORDER BY` sort is not what they think a native would expect. Although we never rule out the possibility that there is a bug, we have found in the past that many people do not read correctly the standard table of weights for the Unicode Collation Algorithm. MySQL uses the table found at http://www.unicode.org/Public/UCA/4.0.0/allkeys-4.0.0.txt. This is not the first table you will find by navigating from the `unicode.org` home page, because MySQL uses the older 4.0.0 “allkeys” table, rather than the more recent 4.1.0 table. This is because we are very wary about changing ordering which affects indexes, lest we bring about situations such as that reported in Bug#16526, illustrated as follows:

    mysql< CREATE TABLE tj (s1 CHAR(1) CHARACTER SET utf8 COLLATE utf8_unicode_ci);
    Query OK, 0 rows affected (0.05 sec)

    mysql> INSERT INTO tj VALUES ('が'),('か');
    Query OK, 2 rows affected (0.00 sec)
    Records: 2  Duplicates: 0  Warnings: 0

    mysql> SELECT * FROM tj WHERE s1 = 'か';
    +------+
    | s1   |
    +------+
    | が  |
    | か  |
    +------+
    2 rows in set (0.00 sec)

The character in the first result row is not the one that we searched for. Why did MySQL retrieve it? First we look for the Unicode code point value, which is possible by reading the hexadecimal number for the `ucs2` version of the characters:

    mysql> SELECT s1, HEX(CONVERT(s1 USING ucs2)) FROM tj;
    +------+-----------------------------+
    | s1   | HEX(CONVERT(s1 USING ucs2)) |
    +------+-----------------------------+
    | が  | 304C                        |
    | か  | 304B                        |
    +------+-----------------------------+
    2 rows in set (0.03 sec)

Now we search for `304B` and `304C` in the `4.0.0 allkeys` table, and find these lines:

    304B  ; [.1E57.0020.000E.304B] # HIRAGANA LETTER KA
    304C  ; [.1E57.0020.000E.304B][.0000.0140.0002.3099] # HIRAGANA LETTER GA; QQCM

The official Unicode names (following the “#” mark) tell us the Japanese syllabary (Hiragana), the informal classification (letter, digit, or punctuation mark), and the Western identifier (`KA` or `GA`, which happen to be voiced and unvoiced components of the same letter pair). More importantly, the primary weight (the first hexadecimal number inside the square brackets) is `1E57` on both lines. For comparisons in both searching and sorting, MySQL pays attention to the primary weight only, ignoring all the other numbers. This means that we are sorting `が` and `か` correctly according to the Unicode specification. If we wanted to distinguish them, we'd have to use a non-UCA (Unicode Collation Algorithm) collation (`utf8_unicode_bin` or `utf8_general_ci`), or to compare the `HEX()` values, or use `ORDER BY CONVERT(s1 USING sjis)`. Being correct “according to Unicode” isn't enough, of course: the person who submitted the bug was equally correct. We plan to add another collation for Japanese according to the JIS X 4061 standard, in which voiced/unvoiced letter pairs like `KA`/`GA` are distinguishable for ordering purposes.

26.12.16: Why don't CJK strings sort correctly in Unicode? (II)

If you are using Unicode (`ucs2` or `utf8`), and you know what the Unicode sort order is (see Section A.12, “MySQL 5.0 FAQ — MySQL Chinese, Japanese, and Korean Character Sets”), but MySQL still seems to sort your table incorrectly, then you should first verify the table character set:

    mysql> SHOW CREATE TABLE t\G
    ******************** 1. row ******************
    Table: t
    Create Table: CREATE TABLE `t` (
    `s1` char(1) CHARACTER SET ucs2 DEFAULT NULL
    ) ENGINE=MyISAM DEFAULT CHARSET=latin1
    1 row in set (0.00 sec)

Since the character set appears to be correct, let's see what information the `INFORMATION_SCHEMA.COLUMNS` table can provide about this column:

    mysql> SELECT COLUMN_NAME, CHARACTER_SET_NAME, COLLATION_NAME
        -> FROM INFORMATION_SCHEMA.COLUMNS
        -> WHERE COLUMN_NAME = 's1'
        -> AND TABLE_NAME = 't';
    +-------------+--------------------+-----------------+
    | COLUMN_NAME | CHARACTER_SET_NAME | COLLATION_NAME  |
    +-------------+--------------------+-----------------+
    | s1          | ucs2               | ucs2_general_ci |
    +-------------+--------------------+-----------------+
    1 row in set (0.01 sec)

(See Section 20.3, “The `INFORMATION_SCHEMA COLUMNS` Table”, for more information.)

You can see that the collation is `ucs2_general_ci` instead of `ucs2_unicode_ci`. The reason why this is so can be found using `SHOW CHARSET`, as shown here:

    mysql> SHOW CHARSET LIKE 'ucs2%';
    +---------+---------------+-------------------+--------+
    | Charset | Description   | Default collation | Maxlen |
    +---------+---------------+-------------------+--------+
    | ucs2    | UCS-2 Unicode | ucs2_general_ci   |      2 |
    +---------+---------------+-------------------+--------+
    1 row in set (0.00 sec)

For `ucs2` and `utf8`, the default collation is “general”. To specify a Unicode collation, use `COLLATE ucs2_unicode_ci`.

26.12.17: Why are my supplementary characters rejected by MySQL?

MySQL does not support supplementary characters — that is, characters which need more than 3 bytes — for `UTF-8`. We support only what Unicode calls the Basic Multilingual Plane / Plane 0. Only a few very rare Han characters are supplementary; support for them is uncommon. This has led to reports such as that found in Bug#12600, which we rejected as “not a bug”. With `utf8`, we must truncate an input string when we encounter bytes that we don't understand. Otherwise, we wouldn't know how long the bad multi-byte character is.

One possible workaround is to use `ucs2` instead of `utf8`, in which case the “bad” characters are changed to question marks; however, no truncation takes place. You can also change the data type to `BLOB` or `BINARY`, which perform no validity checking.

We intend at some point in the future to add support for `UTF-16`, which would solve such issues by allowing 4-byte characters. However, we have as yet set no definite timetable for doing so.

26.12.18: Shouldn't it be “CJKV”?

No. The term “CJKV” (Chinese Japanese Korean Vietnamese) refers to Vietnamese character sets which contain Han (originally Chinese) characters. MySQL has no plan to support the old Vietnamese script using Han characters. MySQL does of course support the modern Vietnamese script with Western characters.

Bug#4745 is a request for a specialized Vietnamese collation, which we might add in the future if there is sufficient demand for it.

26.12.19: Does MySQL allow CJK characters to be used in database and table names?

This issue is fixed in MySQL 5.1, by automatically rewriting the names of the corresponding directories and files.

For example, if you create a database named `楮` on a server whose operating system does not support CJK in directory names, MySQL creates a directory named `@0w@00a5@00ae`. which is just a fancy way of encoding `E6A5AE` — that is, the Unicode hexadecimal representation for the `楮` character. However, if you run a `SHOW DATABASES` statement, you can see that the database is listed as `楮`.

26.12.20: Where can I find translations of the MySQL Manual into Chinese, Japanese, and Korean?

A Simplified Chinese version of the Manual, current for MySQL 5.1.12, can be found at http://dev.mysql.com/doc/#chinese-5.1. The Japanese translation of the MySQL 4.1 manual can be downloaded from http://dev.mysql.com/doc/#japanese-4.1.

26.12.21: Where can I get help with CJK and related issues in MySQL?

The following resources are available:

- A listing of MySQL user groups can be found at http://dev.mysql.com/user-groups/.

- You can contact a sales engineer at the MySQL KK Japan office using any of the following:

      Tel: +81(0)3-5326-3133
      Fax: +81(0)3-5326-3001
      Email: dsaito@mysql.com

- View feature requests relating to character set issues at http://tinyurl.com/y6xcuf.

- Visit the MySQL Character Sets, Collation, Unicode Forum. We are also in the process of adding foreign-language forums at http://forums.mysql.com/.

## A.13. MySQL 5.0 FAQ — Connectors & APIs

For common questions, issues, and answers relating to the MySQL Connectors and other APIs, see the following areas of the Manual:

- Section 22.2.13, “Common Questions and Problems When Using the C API”

- Section 22.3.1, “Common Problems with MySQL and PHP”

- Section 23.1.6, “Connector/ODBC Notes and Tips”

- Section 23.2.5, “Connector/NET Notes and Tips”

- Section 23.4.5, “Connector/J Notes and Tips”

- Section 23.5.5, “Connector/MXJ Notes and Tips”

Prev |  | Next
Chapter 24. Extending MySQL | Home | Appendix B. Problems and Common Errors