Maliit: The only real on-screen keyboard

General, , , ,

I recently pushed the Maliit team here at Openismus to tidy up the website, to make the current status clearer. Here is a summary:

Maliit is the only open-source input system that offers the features real products need, allowing on-screen keyboards for all the world’s languages. It is mature but it needs more work. This is the right project at the right time for various desktop projects to work together instead of having separate hacks that don’t please users.

Everybody loves the virtual keyboard on the N9, which Openismus worked on. That is the Meego Keyboard, which is built on the Maliit framework. It performs. It’s largely open-source, but it is fine-tuned for the N9’s MeeGo 1.2 Harmattan. It has an unfortunate dependency on libmeegotouch, which few developers love.

The newer Maliit Keyboard does not depend on libmeegotouch, though it does depend on Qt and QML. With enough effort, the Maliit Keyboard could be as good as the MeeGo Keyboard.

The underlying Maliit framework API has been thoroughly tested and iterated by these keyboards and by other input methods, both open source and proprietary. It has had many contributors, has gained many regression tests, and is run by people who care deeply about quality.

Note that Maliit is not trying to be an on-screen keyboard for accessibility.

GNOME

We’d like GNOME to allow use of Maliit so people are not forced to use the current Caribou keyboard that only works with GTK+ applications and doesn’t offer features such as CJK or word prediction. We know that GNOME doesn’t want a hard Qt dependency. Of course that would be silly. We do want cross-platform compatibility. We also think that this is a large problem space that GNOME cannot tackle alone.

That would need gnome-shell (or its St toolkit) to support GtkIMContext as Maliit uses it. However, the gnome-shell developers currently feel that any on-screen keyboard (or input method UI, I think) must be part of gnome-shell’s own source code, and can’t be a separate process. Even if caribou could be made to work with non-GTK+ applications, a gnome-shell dependency is an even bigger block to cross-desktop adoption than Qt. So I really hope that this hard problem can be solved, though I do recognize that it’s hard.

The IBus implementation in gnome-shell seems to be hitting several similar issues (1, 2, 3), impacting CJK users even without a virtual keyboard. When they are fixed, maybe Maliit will work better. Note that, though Fedora has replaced SCIM with IBus, neither are really an issue for the on-screen keyboard, because IBus doesn’t let you show windows such as keyboards. However, the non-UI engines in the IBus plugins could (and probably should) be used in Maliit plugins.

I have some more notes about the current gnome-shell virtual keyboard here.

Unity (Ubuntu)

We’d like Ubuntu to use Maliit. We don’t know of any problems with that now, though I guess this depends on the GtkIMContext support in Unity’s nux toolkit library. We will do some tests on the WeTab soon.

Update: We published some Maliit packages for Ubuntu Oneiric in our PPA and we were pleased to find that nux seems to support GtkIMContext so you can, for instance, use the Maliit keyboard in the Dash’s search field. There are a few problems (1, 2, 3) but they seem fixable. There are also some generic GTK+ problems, but these are not blocked by anything fundamental in Unity.

Update 2 much later: That must have been with Unity 2D, using Qt. Nux doesn’t have an input method abstraction as of right now, though it could.

Ubuntu currently ships Onboard, though that’s apparently just for accessibility.

KDE and Qt

We’d like KDE to use Maliit for their tablet efforts. The Maliit keyboards work today in Qt and KDE applications, though I don’t know if that means that they work in Plasma too. They have a Plasma keyboard (source code) already, and there seems to be interest in using Maliit instead of, or as a base for that. That would give it features that it apparently lacks, while also sharing the maintenance workload.

20 thoughts on “Maliit: The only real on-screen keyboard”

  1. First, a bit of hopefully constructive criticism: saying, “Maliit is the only open-source input system that offers the features real products need, allowing on-screen keyboards for all the world’s languages.” is an awesome way to make people working on other projects get really defensive. you’re basically saying “Your projects are not good enough” as a blanket statement without going through the process of describing the why (and hopefully letting people arrive there on their own). When that’s part of an effort to get people to work together … that is more than a little confusing and, to be honest, off-putting as to what it would mean to actually work with and depend on Maliit.

    Ok, ’nuff of that .. ;) the Plasma keyboard is used not only on the tablet but also on the desktop, netbook and whatever other form factors we may target in future. We know it has various issues, but it also works “well enough” to make most people happy at the moment. So the pain threshhold is fairly low (though certainly well above zero), and our resources are stretched (no surprise there, I’m sure :).

    I’ve actually tried Maliit (from git) a few times and it also has its issues. So I know that we’d end up with more work (versus “no work”) by adopting it. Which means there needs to be real upside to it, and it has to provide a real long term reduction in effort …

    What I do like about it is that it works with all software toolkits out there, has advanced things like nice localization support and has a QML based implementation. What I don’t like about it is that it *requires* compositing (which makes it a non-starter on Plasma Desktop, though it’s a non-issue for Plasma Active) and, well, there were numerous quirks I ran into when using it last time (~1 month ago at this point), which I described in a comment on my own blog posting about keyboards.

    In the end, if we can work together, that’d be great. However, instead of trying to convince everyone to come to Maliit and hope it happens .. give us a way to come together. Organize a meeting on IRC where we can get together, learn what everyone needs and discuss how to best move forward (or come to an understanding why we can’t meet each others’ needs, if that is the case). As it is, this blog post doesn’t provide much in the way of an actionable path forward .. just an admonition for people to use Maliit and your desire for others to use your work (which is probably an obvious statement that goes without saying ;)

    I’m on a business trip for the next 5 days, though I will have Internet (or so I desperately hope ;) .. but after that I should be available. Let’s find each other online and try to arrange something like a useful online meeting that we can publicize and get attended by those with interest. Sound cool?

    1. > you’re basically saying “Your projects are not good enough”

      Well, it’s what we think and we think it’s important and we want your attention. However, you should have seen the first draft.

      I have no great expectation of the desktops officially adopting us, simply because that requires overcoming the momentum of their own small solutions. I do, however, hope we can make sure that Maliit works everywhere so it can succeed and become the de-facto system used by distros and device manufacturers. Once it has proven itself, I then believe that the desktop projects will happily not have to work on their own less-popular alternatives. That seems like the path of least resistance. But there are many things that can go wrong even with that plan.

      > without going through the process of describing the why

      Well, my premise here is clearly that an on-screen keyboard isn’t good enough if it doesn’t support multi-toolkits, CJK and word-prediction. And that’s quite apart from other stuff that we’ve found essential such as multitouch support and performance in general. You are welcome to disagree with us about what’s important, but we are confident that device manufacturers agree.

      You are right that we don’t know enough about Plasmaboard. We’d like to know more – particularly if it does indeed support this important stuff.

      > Organize a meeting on IRC where we can get together,

      Sure. I’ll point the Maliit guys here so that can happen. They are motivated. Note that we tried to get involved as soon as we heard about Plasmaboard:
      http://aseigo.blogspot.com/2011/09/on-screen-keyboard.html?showComment=1320687061980#c5968274333245865219
      (Thanks for replying again there yesterday).

      1. > “I do, however, hope we can make sure that Maliit works everywhere so it can succeed and become the de-facto system used by distros”

        Except distros tend to try and stick as close as possible to upstream. So if upstream $DESKTOP use their own on-screen keyboard, it is likely that distros won’t change that.

        I think Aaron is spot on: you are proposing something that you believe is better (and you might very well be right, I frankly have no idea), so it is your job to try and convince others, and that will be more difficult with the words you chose in this article.

        One thing that I don’t understand though, is the argument about the fact that « the gnome-shell developers currently feel that any on-screen keyboard (or input method UI, I think) must be part of gnome-shell’s own source code, and can’t be a separate process. » Can’t Maliit be made into a library, so that desktop environments can build their own simple UI on top of it?

        This way, Gnome-Shell would have its in-process on-screen keyboard written with the Shell Toolkit, Plasma could build its on-screen keyboard UI with their own equivalent, and everyone would be happy? :)

        I have to admit I’m not even a user of on-screen keyboards (no smartphone, no tablet), so I certainly have no idea about the difficulties this entails.

        1. > Except distros tend to try and stick as close as possible to upstream. So if upstream $DESKTOP use their own on-screen keyboard, it is likely that distros won’t change that.

          I don’t think that’s particularly true, particularly not when the upstream solution is not mature or obviously not good enough. It’s not a viable option for distros that care about the features mentioned here. Distros generally ship openoffice/libreoffice instead of GNOME Office. Ubuntu already ship onboard instead of GOK. Distros have plenty of history of making their own major changes.

          > Can’t Maliit be made into a library, so that desktop environments can build their own simple UI on top of it?

          Possibly, but a) We would not ask GNOME to accept a Qt dependency, and b) That would be a lot of work that we don’t expect that they would want to do.

    2. > What I don’t like about it is that it *requires* compositing (which makes it a non-starter on Plasma Desktop, though it’s a non-issue for Plasma Active)

      Support for X without compositing will be available as a side effect of our ongoing work for a good Wayland support in Maliit.

      > Organize a meeting on IRC where we can get together,

      Sounds good.

  2. Just a question:
    “Note that Maliit is not trying to be an on-screen keyboard for accessibility.”: taking into account that Maliit developers were pushing to have Maliit as the on-screen-keyboard for GNOME Shell (so GNOME in general). Are you proposing that GNOME ecosystem should have two on-screen-keyboards? The “real” one, and other specific for accessibility?

    And a comment:
    “… o people are not forced to use the current Caribou keyboard … doesn’t offer features such as CJK or word prediction.”

    FWIW, Caribou can also have word prediction, not built-in, but based on presage:
    http://dpellicer.warp.es/?p=6

    1. > “Note that Maliit is not trying to be an on-screen keyboard for accessibility.”: taking into account that Maliit developers were pushing
      > to have Maliit as the on-screen-keyboard for GNOME Shell (so GNOME in general).

      I don’t think we were. I’m not. I would not propose a hard Qt dependency for GNOME. I’m not delusional.

      > Are you proposing that GNOME ecosystem should have two on-screen-keyboards? The “real” one, and other specific for accessibility?

      I don’t know. I would be interested to hear what Accessibility people think. I just know that Maliit is not attempting to be for Accessibility. On the other hand, it’s not trying to not be for Accessibility either. I’m just stating its current focus.

      I don’t think that’s a problem.

  3. Im not sure this is the right thing for gnome. Openismus is nice and you guys might have experience with on screen keyboards. As a gnome user I would love to see you guys work on this but within gnome. How about you guys propose some improvements to gnomes on screen keyboard and coordinate with the gnomeshell developers?

    I dont ask you to work for free. Im willing to put my money where my mouth is and I believe others feel the same way. Agree on a task with the gnomeshell developers , put a plegie and wait for contributors to show you som love and revenue. What do you think about such an idea Murray?

    1. I don’t think that you have understood that this is about making it possible to work with GNOME, and eventually reducing work for GNOME. A GNOME-only on-screen keyboard is just not good enough, whether it’s the current one or one that someone else develops.

  4. From a recent LWN article about KDE’s Plasma Active One :

    “A touchscreen keyboard is available for text entry, though it suffers from many of the same annoyances that other implementations do. It sometimes pops up in places that cover the text entry field—as it did for searching applications in the screen shot at left—though it can be shifted to the top or bottom of the screen using the arrow icon. It is not really suitable for anything other than entering short text. That’s a limitation of the tablet form factor, but the Ideapad does have a hardware keyboard which makes things a bit easier. The on-screen keyboard does show each key value as it is pressed, but the reaction time seemed slow. It also doesn’t always pop up or disappear at the right times. For someone whose normal interface to a computer is largely through the keyboard, it was irritating, but that may be partially due to the curmudgeon effect.” [The author of the article is wrong though when he reduces the input widget relocation problem to the tablet form factor, it just takes a lot of effort to get it right, see http://wiki.maliit.org/Documentation#Technical_documentation%5D

    This the kind of annoyance I would like to see fixed, and it’s exactly the kind of user experience we (as in: the Maliit developers) were able to polish in Nokia’s N9.

    Writing the virtual keyboard widget itself is only the beginning of a very long journey; I wish more of the involved developers would understand that.

  5. If Maliit was around when I was working on replacing the Gnome On-screen Keyboard, I would have definitely considered it as an option instead of starting Caribou. Yes, it would mean that there would be no “buy-in” from GNOME but I know that writing and maintaining an onscreen keyboard is a lot of work. Too much work, in fact, for the number of accessibility developers who work on open source accessibility keyboard technology. With Caribou, I was hoping to create something that had uses beyond accessibility to attract developers from outside of the accessibility community.

    Here’s the definitive guide to implementing switch based access using a virtual keyboard:

    http://www.ace-centre.org.uk/assets/Product%20Downloads/SwitchScanningMaster_8_472.pdf

    A lot of the stuff in that document (like accessing the switch hardware, debouncing the switch events etc.) would be developed by the AT (Assistive Technology) developers but you can get an idea of the kind of things required. An important thing is key highlighting; there should be a way to highlight: rows, columns, square groups keys for quartering etc.

    Switch access isn’t the only thing people mean when they talk about accessibility and onscreen keyboards but it’s what I had planned to focus on with Caribou. Other groups of people that can use an accessibility oriented onscreen keyboard are: people with cognitive/language impairments and people with visual impairments. Serving people with cognitive/language impairments is usually about being able to use pictures instead of letters to output words or sentences. Serving people with visual impairments is about giving users the ability to adjust the font, font size, font colour, key size, key colour and the space between keys.

    This is just off the top of my head so there’s probably more that could be added. I hope it’s useful.

  6. Since tablets are relatively rare among Ubuntu users, there is little reason for Ubuntu to switch to a screen keyboard that is an accessibility regression. Onboard is actually a pretty good keyboard and is actively maintained. Onboard supports far more keyboard layouts than Caribou and is a lot more usable than Caribou Antler (that’s the standalone version you can use everywhere except GNOME Shell).

    Fortunately for you, Ubuntu now ships Qt on the main CD so that’s one less hurdle you’ll need to pass.

  7. > Since tablets are relatively rare among Ubuntu users,

    The point is that it shouldn’t stay that way. Canonical apparently have big plans to target tablets and other mobile devices.

    > there is little reason for Ubuntu to switch to a screen keyboard that is an accessibility regression.

    Of course. I would never suggest a regression and I would never suggest removing accessibility support. Obviously the two keyboards must remain in parallel, with suitable defaults for suitable devices or users. It would be good to have one true system that does everything, but that’s not practical right now.

    1. Except that most distros only ship one app for each purpose so including two on screen keyboards doesn’t look likely.

      I believe GNOME doesn’t really allow users to choose between multiple installed screen keyboards very well either so you might want to look at getting that improved.

  8. I would love to see an ARM compilation of Maliit in Ubuntu or Debian, so that I could use it in a chroot on my HP TouchPad (WebOS). Because right now the other options for Linux on-screen keyboards available in the ARM repositories are terrible.

    Here’s an example of the Florence keyboard on my TouchPad (in LXDE): http://www.flickr.com/photos/qole2/6107849155

  9. Of course people can compile it themselves, but you won’t get as many users that way!
    I would love to see arm builds on the ppa beside the x86 builds…

  11. Pingback: Murray’s Blog » Blog Archive » Openismus at FOSDEM 2012

Comments are closed.