Support IME contextual conversion

[tats-u]

MS-IME supports IME contextual conversion. By this feature, MS-IME can suggest correct candidate 鼻 and 花 from はな according to surrounding characters at the end of the following lines:

象の
可憐な

Copy the text and paste it on Notepad (never VS Code) or some <textarea> in a browser first.

Mozc and Google Japanese Input cannnot.

From: microsoft/vscode#233641

Note: the following descriptions are all editor side!

• IMEの前後参照変換機能に対応するには: This article provides an overview of how applications can support IME's contextual conversion by utilizing the IMR_DOCUMENTFEED notification code.
• TextBox を IME 前後参照変換に対応させる (System.Windows.Forms): This post discusses implementing IME contextual conversion in a Windows Forms TextBox control, including sample code and explanations.

Hidemaru's changelog: http://hide.maruo.co.jp/software/hidemaru7/new9.html

[yukawa]

This has been in our TODO list for long time.

The easiest part, which is forwarding the surrounding text from the application to mozc_server, was already achieved back in 1.11.1490.10x.

• docs/design_doc/surrounding_text.md
• Google 日本語入力の開発版をアップデートしました（1.11.1490.10x）

Here is an example of the case when mozc_server changes the output depending on the surrounding text as of Mozc-2.31.5712.100+24.11.oss.

You should be able to observe the same behavior on Windows, macOS, and Linux as long the target application correctly supports surrounding text IME API. Note that here is the relevant logic in mozc_server on where the above behavior difference is coming from.

mozc/src/engine/engine_converter.cc

Lines 1661 to 1706 in 76343ff

	const std::string &preceding_text = context.preceding_text();
	// If preceding text is empty, it is OK to reset the history segments by
	// calling ResetConversion.
	if (preceding_text.empty()) {
	converter_->ResetConversion(&segments_);
	return;
	}
	// Hereafter, we keep the existing history segments as long as it is
	// consistent with the preceding text even when revision_changed is true.
	std::string history_text;
	for (const Segment &segment : segments_) {
	if (segment.segment_type() != Segment::HISTORY) {
	break;
	}
	if (segment.candidates_size() == 0) {
	break;
	}
	history_text.append(segment.candidate(0).value);
	}
	if (!history_text.empty()) {
	// Compare |preceding_text| with |history_text| to check if the history
	// segments are still valid or not.
	DCHECK(!preceding_text.empty());
	DCHECK(!history_text.empty());
	if (preceding_text.size() > history_text.size()) {
	if (absl::EndsWith(preceding_text, history_text)) {
	// History segments seem to be consistent with preceding text.
	return;
	}
	} else {
	if (absl::EndsWith(history_text, preceding_text)) {
	// History segments seem to be consistent with preceding text.
	return;
	}
	}
	}
	// Here we reconstruct history segments from |preceding_text| regardless
	// of revision mismatch. If it fails the history segments is cleared anyway.
	if (!converter_->ReconstructHistory(&segments_, preceding_text)) {
	LOG(WARNING) << "ReconstructHistory failed.";
	DLOG(WARNING) << "preceding_text: " << preceding_text
	<< ", segments: " << segments_.DebugString();
	}

While I am not sure if the Mozc team has some time to work on this area in the near term future, the good news is that if you have some implementation ideas and are interested in implementing it in Mozc, you can just start implementing it in mozc_server without being bothered by researching how to retrieve surrounding text for each platform.

Hope this helps.