JNI based show_soft_input

[jancespivo]

Hi,
this PR is trying to solve not showing soft keyboard on Android.

It might solve #44 🤷

I've tested it on Android version 10 and 12 with xilem (main + winit 0.30.4 (with cherrypicked rust-windowing/winit#3993).

rust-windowing/winit#3787 doesn't work for me at all even it can be better approach for some Android versions.

I've intentionally kept hide_soft_input as is, it works.

[podusowski]

I think the main problem with the keyboard not showing up on NativeActivity is its View not being in focus. I have hacked the way through by using something like this:

public class MainActivity extends NativeActivity implements OnApplyWindowInsetsListener {
  View getNativeActivityView() {
    // This is hacky as hell, but NativeActivity does not give any proper way of
    // accessing it.
    ViewGroup parent = (ViewGroup) (getWindow().getDecorView());
    parent = (ViewGroup) parent.getChildAt(0);
    parent = (ViewGroup) parent.getChildAt(1);
    return parent.getChildAt(0);
  }

  @Override
  protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    View nativeActivityView = getNativeActivityView();
    nativeActivityView.setFocusable(true);
    nativeActivityView.setFocusableInTouchMode(true);
    nativeActivityView.requestFocus();

    ViewCompat.setOnApplyWindowInsetsListener(nativeActivityView, this);

but ultimately switched to GameActivity anyway because of other issues.

[jb55]

but ultimately switched to GameActivity anyway because of other issues.

what issues?

[podusowski]

I haven't done any Android programming before, so maybe it's just me not knowing my way around, but I wasn't able to make the view to respect insets. So the status bar, navigation bar and keyboard were all drawn above my app.

[podusowski]

On the other hand, I can't make the IME on GameActivity to send me raw key events. Instead, it has its own input handling which ultimately sends IME events and expects feedback. These however are not supported by the winit/egui. There was some grat work done by @lucasmerlin, but afaik it didn't get through. Basically I hit a dead end with the Android keyboard, and made this for now. :D

[jancespivo]

I would like to push this PR further. It fixes the problem for all my devices. Could you please review it @MarijnS95 ?
I know there could be some more modern way to do it with WindowInsets api, however there is no PR. So this might be a good first step. WDYT?

[MarijnS95]

Is there an existing minimal example (one that could possibly be added to the examples folder in this repository) to demonstrate how IME is working? That would make it significantly easier to review this now and experiment with IME in the future, as I do not currently use nor am familiar with it.

[podusowski]

Current example already opens an IME after clicking on a specific area.

android-activity/examples/na-mainloop/src/lib.rs

Line 101 in 51d05d4

if x < 200.0 && y < 200.0 {

[jb55]

Current example already opens an IME after clicking on a specific area.

android-activity/examples/na-mainloop/src/lib.rs

Line 101 in 51d05d4

if x < 200.0 && y < 200.0 {

weird, when I try this i get:

03-10 22:26:46.618 14299 14299 W InputMethodManager: Ignoring showSoftInput()
as view=android.app.NativeActivity$NativeContentView{9e49b2f V.ED..... ......ID
0,0-1080,2209} is not served.

I get this on master and this branch.

I guess this is because of:

• JNI based show_soft_input #178 (comment)

[wareya]

this actually works for me already, though I had to copy it into my application

showSoftInput can trigger exceptions if the keyboard is currently animating open/closed, so the final method call should be followed up with something like:

if env.exception_check().unwrap() { env.exception_clear(); }

and if you REALLY want things to be clean, make the function return an Option<()>, with None if there was an exception and Some(()) otherwise. the value of the exception isn't particularly useful in this case.

for posterity, the version I have at the moment:

#[cfg(target_os = "android")]
pub fn show_soft_input(show_implicit : bool)
{
    // https://github.com/rust-mobile/android-activity/pull/178
    use std::borrow::Borrow;
    let ctx : egui_winit::winit::platform::android::activity::AndroidApp = unsafe { APP_CONTEXT.as_ref().unwrap().clone() };
    
    let jvm = unsafe { jni::JavaVM::from_raw(ctx.vm_as_ptr().cast()) }.unwrap();
    let na = unsafe { jni::objects::JObject::from_raw(ctx.activity_as_ptr().cast()) };
    
    let mut env = jvm.attach_current_thread().unwrap();
    if env.exception_check().unwrap() { return; }
    let class_ctxt = env.find_class("android/content/Context").unwrap();
    if env.exception_check().unwrap() { return; }
    let ims = env.get_static_field(class_ctxt, "INPUT_METHOD_SERVICE", "Ljava/lang/String;").unwrap();
    if env.exception_check().unwrap() { return; }
    
    let im_manager = env.call_method(
        &na, "getSystemService", "(Ljava/lang/String;)Ljava/lang/Object;", &[(&ims).into()]
    ).unwrap().l().unwrap();
    if env.exception_check().unwrap() { return; }
    
    let jni_window = env.call_method(&na, "getWindow", "()Landroid/view/Window;", &[]).unwrap().l().unwrap();
    if env.exception_check().unwrap() { return; }
    let view = env.call_method(&jni_window, "getDecorView", "()Landroid/view/View;", &[]).unwrap().l().unwrap();
    if env.exception_check().unwrap() { return; }
    
    env.call_method(
        im_manager,
        "showSoftInput",
        "(Landroid/view/View;I)Z",
        &[
            jni::objects::JValue::Object(&view),
            if show_implicit
            {
                (ndk_sys::ANATIVEACTIVITY_SHOW_SOFT_INPUT_IMPLICIT as i32).into()
            }
            else
            {
                0i32.into()
            },
        ],
    );
    // showSoftInput can trigger exceptions if the keyboard is currently animating open/closed
    if env.exception_check().unwrap() { env.exception_clear(); }
}