The built-in MySQL full-text parser uses the white space between words as a delimiter to determine where words begin and end, which is a limitation when working with ideographic languages that do not use word delimiters. To address this limitation for Japanese, MySQL provides a MeCab full-text parser plugin. The MeCab full-text parser plugin is supported for use withInnoDBandMyISAM.

The MeCab full-text parser plugin is a full-text parser plugin for Japanese that tokenizes a sequence of text into meaningful words. For example, MeCab tokenizes“データベース管理”(“Database Management”) into“データベース”(“Database”) and“管理”(“Management”). By comparison, thengramfull-text parser tokenizes text into a contiguous sequence ofncharacters, wherenrepresents a number between 1 and 10.

In addition to tokenizing text into meaningful words, MeCab indexes are typically smaller than ngram indexes, and MeCab full-text searches are generally faster. One drawback is that it may take longer for the MeCab full-text parser to tokenize documents, compared to the ngram full-text parser.

The full-text search syntax described inSection 12.10, “Full-Text Search Functions”applies to the MeCab parser plugin. Differences in parsing behavior are described in this section. Full-text related configuration options are also applicable.

For additional information about the MeCab parser, refer to theMeCab: Yet Another Part-of-Speech and Morphological Analyzerproject on Github.

Installing the MeCab Parser Plugin

The MeCab parser plugin requiresmecabandmecab-ipadic.

On supported Fedora, Debian and Ubuntu platforms (except Ubuntu 12.04 where the systemmecabversion is too old), MySQL dynamically links to the systemmecabinstallation if it is installed to the default location. On other supported Unix-like platforms,libmecab.sois statically linked inlibpluginmecab.so, which is located in the MySQL plugin directory.mecab-ipadicis included in MySQL binaries and is located inMYSQL_HOME\lib\mecab.

You can installmecabandmecab-ipadicusing a native package management utility (on Fedora, Debian, and Ubuntu), or you can buildmecabandmecab-ipadicfrom source. For information about installingmecabandmecab-ipadicusing a native package management utility, seeInstalling MeCab From a Binary Distribution (Optional). If you want to buildmecabandmecab-ipadicfrom source, seeBuilding MeCab From Source (Optional).

On Windows,libmecab.dllis found in the MySQLbindirectory.mecab-ipadicis located inMYSQL_HOME/lib/mecab.

To install and configure the MeCab parser plugin, perform the following steps:

Creating a FULLTEXT Index that uses the MeCab Parser

To create aFULLTEXTindex that uses the mecab parser, specifyWITH PARSER ngramwithCREATE TABLE,ALTER TABLE, orCREATE INDEX.

This example demonstrates creating a table with amecabFULLTEXTindex, inserting sample data, and viewing tokenized data in theINFORMATION_SCHEMA.INNODB_FT_INDEX_CACHEtable:

mysql> USE test; mysql> CREATE TABLE articles ( id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY, title VARCHAR(200), body TEXT, FULLTEXT (title,body) WITH PARSER mecab ) ENGINE=InnoDB CHARACTER SET utf8; mysql> SET NAMES utf8; mysql> INSERT INTO articles (title,body) VALUES ('データベース管理','このチュートリアルでは、私はどのようにデータベースを管理する方法を紹介します'), ('データベースアプリケーション開発','データベースアプリケーションを開発することを学ぶ'); mysql> SET GLOBAL innodb_ft_aux_table="test/articles"; mysql> SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_INDEX_CACHE ORDER BY doc_id, position;

To add aFULLTEXTindex to an existing table, you can useALTER TABLEorCREATE INDEX. For example:

CREATE TABLE articles ( id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY, title VARCHAR(200), body TEXT ) ENGINE=InnoDB CHARACTER SET utf8; ALTER TABLE articles ADD FULLTEXT INDEX ft_index (title,body) WITH PARSER mecab; # Or: CREATE FULLTEXT INDEX ft_index ON articles (title,body) WITH PARSER mecab;

MeCab Parser Space Handling

The MeCab parser uses spaces as separators in query strings. For example, the MeCab parser tokenizesデータベース管理asデータベースand管理.

MeCab Parser Stopword Handling

By default, the MeCab parser uses the default stopword list, which contains a short list of English stopwords. For a stopword list applicable to Japanese, you must create your own. For information about creating stopword lists, seeSection 12.10.4, “Full-Text Stopwords”.

MeCab Parser Term Search

For natural language mode search, the search term is converted to a union of tokens. For example,データベース管理is converted toデータベース 管理.

SELECT COUNT(*) FROM articles WHERE MATCH(title,body) AGAINST('データベース管理' IN NATURAL LANGUAGE MODE);

For boolean mode search, the search term is converted to a search phrase. For example,データベース管理is converted toデータベース 管理.

SELECT COUNT(*) FROM articles WHERE MATCH(title,body) AGAINST('データベース管理' IN BOOLEAN MODE);

MeCab Parser Wildcard Search

Wildcard search terms are not tokenized. A search onデータベース管理*is performed on the prefix,データベース管理.

SELECT COUNT(*) FROM articles WHERE MATCH(title,body) AGAINST('データベース*' IN BOOLEAN MODE);

MeCab Parser Phrase Search

Phrases are tokenized. For example,データベース管理is tokenized asデータベース 管理.

SELECT COUNT(*) FROM articles WHERE MATCH(title,body) AGAINST('"データベース管理"' IN BOOLEAN MODE);

Installing MeCab From a Binary Distribution (Optional)

This section describes how to installmecabandmecab-ipadicfrom a binary distribution using a native package management utility. For example, on Fedora, you can use Yum to perform the installation:

yum mecab-devel

On Debian or Ubuntu, you can perform an APT installation:

apt-get install mecab apt-get install mecab-ipadic

Installing MeCab From Source (Optional)

If you want to buildmecabandmecab-ipadicfrom source, basic installation steps are provided below. For additional information, refer to the MeCab documentation.