Home > Keyboard Release, Keyboards > Starting to Fully Optimize the Keyboard

Starting to Fully Optimize the Keyboard

(Edit: I found a bug in the way rolls were being calculated. MTGAP 0.1 (shown below) is no longer the best layout.)

It’s been a while since I’ve done anything with the New Keyboard Layout Project, but I read a comment on one of my posts and I got to thinking about punctuation. Every keyboard I’ve designed has just been based on the main 30 keys and used .,’; as the four punctuation marks, because those are the ones that Dvorak used. But why use those four punctuation marks? Why not use a different set?

In fact, why not simply try to optimize the entire keyboard instead of just the main 30 keys?

Previously, the answer to that question was that it would be too slow. But now, thanks to a much-improved algorithm, I no longer have that excuse. That means I can evaluate the entire keyboard.

The Physical Keyboard

Changing the size of the keyboard requires rewriting large portions of the program. For this reason I didn’t want to rewrite it for a standard physical keyboard — why design such a highly optimized layout for a suboptimal physical keyboard? Instead, I rewrote the program to optimize on the Kinesis Advantage Pro keyboard. (You can see a good picture of the full keyboard here.) I ignored the thumb pads, tab, shift and caps lock for aesthetic reasons, and the arrow keys and function keys because it is nearly impossible to determine the frequency of those keys. This leaves four rows and 47 keys. The QWERTY keyboard looks like this:

1 2 3 4 5 6 7 8 9 0 -
Q W E R T Y U I O P \
A S D F G H J K L ; '
Z X C V B N M , . /
` =             [ ]

Shifted Keys

My program doesn’t deal with shifted keys, and modifying it to do so would be a much greater task than what I am currently doing. Rather than try to get the program to deal with shifted keys, I decided to simply choose the most common punctuation and put those on the unshifted slots.

There are 26 letters and 10 numbers. Out of 47 spots this leaves 11 spots for punctuation. The 11 most common punctuation marks are:

, . ) ( _ \ " ; - ' = /

So I pulled off the standard punctuation and stuck those on.

Results

After some (not insignificant) modifications, the program was able to optimize a full-sized keyboard. You can download my earliest functional version of the program here. It hasn’t been extensively tested and it’s messy, but it’s functional.

The first layout it came up with was this one:

Hands: 53% 46%
Fingers: 10% 10% 10% 21% 13% 14% 10% 8%


x 3 6 5 q / " 9 2 8 0
u l c o ; v m d p ) j
a r s e , f h t n i -
( ' w . = k y g b _
4 1             z 7

Fitness: 20014435648
Distance: 97925496
Inward rolls: 16.44%
Outward rolls: 5.40%
Same hand: 48.05%
Same finger: 2.02%
Row change: 21.05%
Home jump: 0.80%
To center: 3.03%
To outside: 0.40%

(I’ve added a new cost: “to outside.” It’s similar to “to center”: it penalizes a layout every time the user has to reach to the outside of the keyboard with the pinky before or after typing a letter on that same hand.)

Highly optimized and aesthetically horrible. The number keys, instead of being in a nice straight line, are all over the place. The parentheses aren’t even next to each other. This isn’t much of an issue when you’re dealing with the main 30 characters because there are no real aesthetics to speak of, but once you expand it becomes a serious problem.

The solution is to require that the program put certain keys in certain places: the number keys on the top row and the parentheses next to each other. There are two fundamental ways to do this: force it to, or give a penalty for not doing so. I found that the best way to keep the number keys in place was simply to tell the computer that it wasn’t allowed to move them. That doesn’t quite work with parentheses though, because they should still be able to move around; they just should stay next to each other. If one moves, the other moves. Forcing them to be next to each other but still be able to move around as a chunk would require adding an extra layer of complexity to the program. The simpler solution is to heavily penalize a keyboard layout every time the parentheses aren’t next to each other.

After adding these restrictions and tweaking the costs a bit, I came up with this layout:

MTGAP Full 0.1

Hands: 52% 47%
Fingers: 9% 10% 18% 13% 13% 14% 10% 9%


1 2 3 4 5 6 7 8 9 0 q
y c o u ( ) l d p w x
i s e a , m h t n r k
_ v " . ; ' f g b -
/ =             z j

Fitness:       193491944
Distance:      956628
Inward rolls:  8.42%
Outward rolls: 2.20%
Same hand: 36.00%
Same finger: 1.48%
Row change: 17.14%
Home jump: 0.26%
To center: 2.29%
To outside: 0.50%

Some of its numbers are quite impressive. For comparison, here’s Colemak (with punctuation modified a bit to fit on the keyboard):

Hands: 46% 53%
Fingers: 8% 8% 11% 18% 18% 15% 10% 9%


1 2 3 4 5 6 7 8 9 0 -
q w f p g j l u y ; =
a r s t d h n e i o '
z x c v b k m , . /
_ "             ( )

Fitness:       230028740
Distance:      1006256
Inward rolls:  4.53%
Outward rolls: 2.62%
Same hand: 42.86%
Same finger: 2.01%
Row change: 18.93%
Home jump: 0.74%
To center: 7.54%
To outside: 0.48%

My layout beats Colemak on every single metric except “to outside” (and possibly outward rolls, depending on whether you like those or not). Notice that, even though my layout puts ‘o’ (the fourth most common letter) off the home row, it still has lower travel distance than Colemak.

(In case you’re new here, the reason I compare my layout to Colemak is because Colemak is my favorite keyboard layout that I didn’t design.)

Also, if you’re interested, here’s Dvorak:

Hands: 44% 55%
Fingers: 8% 8% 12% 14% 16% 13% 13% 11%


7 5 3 1 9 0 2 4 6 8 =
' , . p y f g c r l /
a o e u i d h t n s -
; q j k x b m w v z
_ "             ( )

Fitness:       247807385
Distance:      1020108
Inward rolls:  4.14%
Outward rolls: 1.25%
Same hand: 31.14%
Same finger: 3.16%
Row change: 14.36%
Home jump: 0.50%
To center: 7.39%
To outside: 0.39%

Advertisements
  1. Atle
    January 4, 2011 at 4:47 am

    Your result is similar to the mod layout that Im using!

    Using a ‘standard’ laptop layout, here is what mine looks like.

    ycouj kdmlw z
    isea. fhtnr q
    ()-,’ bpgxv

    It works well.

    • January 4, 2011 at 1:16 pm

      Did you put all the punctuation in one place on purpose, or did it just turn out that way?

      • Atle
        January 4, 2011 at 6:16 pm

        The rational mapping for punctuations is a combination of intention and statistical luck.

        Indeed I wanted punctuations in one easy to cognate area, so I could swap them out for different characters in alternative layouts. Serendipitously, your program already achieved this result by ‘wrapping’ the a-u and e-o keys with punctuations to prevent same-finger sequences.

        I had already moved the parentheses to the QWERTY brackets keys – with better reach without SHIFT – but I noticed I still kept on straining for them. Since I felt I was using parentheses more frequently than x, q, or z, I more-or-less swapped the QWERTY brackets keys for the q-z keys.

        The result is an elegant punctuation map that swathes vowels.

        • Atle
          January 4, 2011 at 7:30 pm

          For non-programming, each parenthesis is compable in frequency to the letter z. However, if Im looking at the statistics the right way, since you always have to use both parentheses, the set together is roughly twice as frequent as the letter z. And I probably use parentheses more than most. In any case, layout is more comfortable for me with q and z to the outside, and the parentheses within general reach.

          • January 5, 2011 at 12:51 am

            Interestingly enough, the close parenthesis actually occurs more often than the open parenthesis (3095538 versus 3094251 — that doesn’t mean that that’s how many parentheses there are because one parenthesis doesn’t always count as one point).

    • Sam
      January 28, 2011 at 9:51 am

      How did you decide on this layout? Did you base it on one of Michaek’s layouts? I like the middle finger usage here, although I have a fondness for bottom row index finger keys.

  2. Atle
    January 4, 2011 at 4:58 am

    I was going to go with the following layout, except I get unusually heavy work out the j.

    ycou, kdmlw qz
    isea. fhtnr x
    ()-‘” bpgjv

  3. Floatharr
    January 4, 2011 at 10:11 am

    Hi!

    One thing that bothers me about all the layouts is that ; and maybe – key are almost never used unless you’re typing formal text, which for for common computer usage is never. Most people don’t even know how to use a semicolon and dashes are in most cases replaced with a space, if a word requiring one even occurs in their vocabulary. Easy to obtain sample texts large enough might have more complex language than what a normal computer user might use, skewing the letter frequencies in addition to the special characters.

    Most people don’t use their keyboard to write books and I doubt even just forum posts and blogs would paint the whole picture. What I think would be necessary instead is to use keylogs from a large set of people with different habits, optimally of just the target audience which I assume would be people with above average knowledge of computers. Things such as URLs, file paths, emoticons, console use, commands and file names also need to be a factor in a modern keyboard specifically designed for a modern user. As to how to obtain large quantities of text generated by every day computer use, short of installing keyloggers to people’s computers, is a whole another matter.

    • Atle
      January 4, 2011 at 6:53 pm

      Heh, your comment about the uselessness of the semicolon vindicates what I said elsewhere. Indeed, for those who dont do code programming, the semicolon is less frequent than the exclamation mark (for me, less frequent than brackets!).

      I do use the dash frequently – not only for hyphenated words and two-word adjectives, but also for the space-dash-space instead of the m-dash or ‘long dash’.

      I think its useful to have different layouts with alternative punctuations, depending on the genre. If I did C-programming every day, I would have a layout that looked something like this:

      ycou' kdmlw qz
      isea; fhtnr x
      ()_*= bpgjv
      

      Thus
      SHIFT; is the ,
      SHIFT* is the .
      SHIFT_ is the –
      SHIFT= is the ”

      I would simultaneously use a normal layout that inverted these SHIFT positions: eg, SHIFT” is = .

  4. Atle
    January 5, 2011 at 6:23 am

    Michael Dickens :Interestingly enough, the close parenthesis actually occurs more often than the open parenthesis (3095538 versus 3094251 — that doesn’t mean that that’s how many parentheses there are because one parenthesis doesn’t always count as one point).

    Hmmm. The two would count as one set that equals the least of either of them. So if you use open, you also use closed too. The extra closed are probably for people who use them for numbering lists: 1) 2) 3) etcetera.

    More than that. Compare ‘smart quotes’ that automatically assign whether to use an open quote character or a close quote character. If you had a ‘smart parentheses’ macro, you would simply use one key for both characters. In this sense you would simply add their percent frequencies together, same as quotes.

  5. Atle
    January 6, 2011 at 4:33 pm

    With regard to the rational mapping for punctuations, even the newest result produces it via statistical luck.

    12345 67890 q
    ycou( )ldpw x
    isea, mhtnr k
    _-".; 'fgbv
    /= zj

    The keys assigned for the punctuation – and the letter v make a fairly neutral swap. The result is a clear swath of punctuations around the vowel letters.

  6. phynnboi
    January 14, 2011 at 8:02 am

    Do you actually own a Kinesis Advantage Pro? I know you’re just doing this for fun and don’t expect anyone to adopt the resultant layouts, but still, you seem to be aiming at a truly minuscule portion of keyboard users, here! Even where I doing it for fun, I’d at least like the facade that my work would be generally applicable, even if no one ever actually applied it.

    Gotta admit, I got a kick out of your program keeping the parentheses “together,” yet still putting each one on a different hand. It’s as if the AI is defying you, probably for the lulz. Better keep an eye on it; who knows what it’ll do next!

    • January 14, 2011 at 1:20 pm

      True. I could always try to optimize for a “standard” (a.k.a. suboptimal) physical keyboard and that would probably end up with a result that’s more accessible to most people. But it’s still no fun to create a layout that’s optimal in every way except that it used a suboptimal physical keyboard.

      I actually don’t mind having the parentheses on opposite hands since they still both correspond to the same position on either hand and they’re next to each other.

      My program is also doing something weird where it removes one of the numbers from its designated spot, but it only does this about one in a hundred thousand times so it’s really hard to test for, and it only removes one number.

      Edit: Okay, you win, I made a layout for the standard keyboard.

  7. Arne Jansen
    January 15, 2011 at 11:34 am

    For years I’m very frustrated with my qwerty-keyboard when doing C-programming. For typing texts it’s good enough for me as I don’t write that often.
    A long while ago I learned dvorak. It really feels nicer for text, but doesn’t help much for programming. So now I’m on the quest to finally find the right layout for me.
    As you stated, most optimizations are only for letters, not for special characters. So the requirements I have are that C-programming (and maybe perl, with quite different frequencies) have to be really easy and fun, while typing text shouldn’t be harder than with qwerty.
    I also want to take one modification into account: As a lot a keys are only reachable with the right pinky and return needs a good stretch, I’d like to shift the right hand position one to the right. That distributes a lot of load from the pinky to both index fingers and makes return, backspace and AltGr much easier to reach.
    Hopefully I can use your program as a starting point!

    • January 16, 2011 at 1:49 am

      I developed a keyboard layout that values C and Perl text more highly:

      Hands: 51% 48%
      Fingers: 10% 9% 18% 13% 13% 14% 10% 9%


      * 1 2 3 4 5 6 7 8 9 0 z $
      y p o u . ; d l c w x j q
      i n e a , f h t s r "
      / = ( ) - v m g b k

      Fitness: 256021317
      Distance: 440948
      Inward rolls: 9.32%
      Outward rolls: 2.28%
      Same hand: 35.12%
      Same finger: 1.99%
      Row change: 14.05%
      Home jump: 1.01%
      To center: 2.73%
      To outside: 1.64%

      It uses these punctuation marks: , . ( ) ; = ” – / $ *

      If you want to shift the hand to the right, that will require some significant modifications. You can download the program here. I’m working on making it more user-friendly. But for now the way to shift the hand to the right is to do these things:

      (If you want to optimize a keyboard for C programming then I assume you know C.)

      1. In initValues() in values.c, look for the definition of costsCopy inside of “else if (full_keyboard == FK_STANDARD)”. The numbers represent the costs of each position on the keyboard, and a 999 means that that key does not exist. It doesn’t actually matter whether the number is 999 as long as there is some number there. You’ll definitely want to change the numbers on the right side of the keyboard, from this


      140, 120, 90, 75, 95, 130, 160, 95, 75, 90, 120, 120, 140, 999,
      999, 70, 40, 30, 30, 60, 80, 30, 30, 40, 70, 90, 110, 150,
      999, 10, 4, 0, 0, 30, 30, 0, 0, 4, 10, 85, 999, 999,
      999, 90, 85, 60, 50, 95, 70, 40, 60, 80, 90, 999, 999, 999,

      to something like this


      140, 120, 90, 75, 95, 130, 190, 160, 95, 75, 90, 120, 120, 999,
      999, 70, 40, 30, 30, 60, 100, 80, 30, 30, 40, 70, 90, 110,
      999, 10, 4, 0, 0, 30, 60, 30, 0, 0, 4, 10, 999, 999,
      999, 90, 85, 60, 50, 95, 130, 70, 40, 60, 80, 999, 999, 999,

      2. In initData() in tools.c, again look for “else if (full_keyboard == FK_STANDARD)”. You’ll see a series of arrays called fingerCopy, handCopy, isCenterCopy, and is OutsideCopy (there are a couple of others but you don’t need to modify them). They all are 4 rows of 14, representing the 47 keys on the keyboard, just like costsCopy. Modify each of those appropriately.

      If anything is unclear or something in the program doesn’t work, let me know.

  8. Stephen O'Connor
    January 11, 2012 at 9:07 pm

    Since you are using the Kinesis which has two “home” thumb buttons available, are you not curious to try placing a single very common letter on one of the thumb buttons? For example, the Maltron keyboard which is similar to the Kinesis has its own Maltron layout that places the letter E on a thumb while the other thumb is used for space as normal. This is conducive to true 10 finger typing rather than just 9, except for the small portion of people who actually press the spacebar with the thumb on the hand opposite the last key that was pressed, which is still not ideal usage of the second thumb. In any case, I think moving one of the more common letters to a thumb would cause a significant impact on the rest of the keyboard layout that would be interesting to observe.

    • Stephen O'Connor
      January 11, 2012 at 9:13 pm

      Additionally, you may be interested in this keyboard as well which finally reached customers’ hands after over a year of production delays which had many people worried. http://www.trulyergonomic.com/

      I own a Kinesis and was thinking of purchasing one of these due to how compact it is (refer to a picture without the wrist rest attached), making it more viable for travel. I would then have a keyboard with two “home” thumb buttons at all times and thought it would be nice to take advantage of it.

      I went from Qwerty to Dvorak to Colemak and use the caps lock key as backspace which is what the Kinesis uses for its other thumb button. However, since good typists strive for very high accuracy (say above 95%), I fell that the thumb could be put to better use and that caps lock remains a good backspace key.

      • January 12, 2012 at 3:27 pm

        Truly Ergonomic looks better than most keyboards, but not as good as Kinesis. I don’t like how it places keys in the center column—it should at the very least break each of those into two keys.

    • January 12, 2012 at 3:26 pm

      I plan on modifying the keyboard layout optimizer to include thumb keys. Until then, it is possible to change the default starting layout (in “kinesislayoutstore.txt”) by replacing E with a different character. As long as that character is included in the character and digraph frequency lists that the program uses, it should work fine.

  9. April 26, 2012 at 7:38 pm

    Dear author, as I am interested in the design of the keyboard layout, may I ask what program you have used to test the layout?

    • April 27, 2012 at 3:07 pm

      I’m not sure what you mean by “test”. I created the optimized keyboard layout with a computer program that may be found here: https://github.com/MTGandP/Typing

      • April 29, 2012 at 3:58 am

        Thank you so much for your quick reply. I am currently doing a math project, regarding to the optimized keyboard layout…but I am kind of stuck now, because I haven’t applied much math stuff and I only consider single letter frequency.
        Could you please give me some advice on which math-related theories I can use to design the layout, taking into consideration bigrams frequency? Acutally I don’t plan to depend too much on software…coz it is a math project…so it would be good to just design it manually 🙂

        Yours faithfully
        Ranson

        • April 30, 2012 at 7:02 pm

          I suggest you try to determine what sorts of finger combinations are difficult. Keyboard layout experts agree on most such issues, and you can find information on this at a number of websites, which I list here: http://mtgap.bilfo.com/alternative_layouts.html

          Then decide how important each criterion is, and build a keyboard layout based on your criteria. It’s probably a good idea to use computer software to get statistics on your layouts. You can use online tools (there are a few good ones, but I don’t have the links on hand) or my own program (https://github.com/MTGandP/Typing). Then revise your layout.

          • May 1, 2012 at 1:44 pm

            Thank you so much for you help and guidance 🙂
            Yours faithfully

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: