The first step of Luna was to outline the behavioral requirements for a conversational AI interaction. At this point, I had not yet defined sleep as the final target, but the core user experience flow remains regardless of behavior. This outline was a direct result of the research I had done with MOTI. One of the key pieces here was having users commit each night to a new bedtime. This accomplished three things, 1) it was more difficult to ignore than an alarm going off at the same time each night, 2) accounted for small fluctuations in when a user could or wanted to go to bed, making Luna feel more empathetic, and 3) forced the user to reflect and make a commitment to themselves, and have an external force hold them accountable for it.

The next step was selecting a framework for conversational development. The type of relationship that I wanted to build with users was too complex for a basic tree-based conversation flow. This ruled out the framework I had previously used with the New Years Resolution Bot, motion.ai.

The biggest factors of this choice included: thee best intent matching, ability to easily train, and an easy way to persist user states.

Our final architecture. We selected API.ai (now DialogueFlow) since it seemed to be the most robust from an NLP perspective and easy to integrate with.

However, API.ai is built for developers, not designers. A list of intent names is no way to design a conversational experience, so I needed to figure out a way to convey to my developers what the bot should actually do.

At the time there was no pre-existing tool for this, so I built a good number of flow chart outlining user messages and bot intents, contexts, and events.

Our approach to user testing was fail early, fail often. We used our friends and family to mechanically turk the QA process in the early days, and attended conferences giving Luna away for free to observe how users reacted to her personality and tone.

In addition to the user experience elements, we continued to gather information on people’s sleep habits and their attitudes about them. Here, the balance was how much information we could get out of conference-goers in one minute or less of engagement!

One feature (of many) that came out of user testing was the concept of morning check ins. This was a great way to built a feedback loop from the behavior of the previous night to how good (or bad) users felt after a night of sleep. The challenging thing with sleep is that going to bed on time isn’t immediately gratifying — you only feel good the next day. Spanning that gap was a psychological hurdle we always sought to overcome.

One of the biggest elements of conversational AI designed I learned was that variation is key. This keeps the user on their toes, and makes the agent feel more human (human are very good at not repeated themselves word-for-word). This was a second of three variations of the morning check-ins flow.

Another element of variation was when to ask people to commit to a bedtime each night. We allowed users to customize when this request would come in, since different users would be home from work or the gym or at dinner at different times. However, we tried not to send Luna’s pokes right at a specific UTC, but vary it +/- a few minutes to prevent users from being able to predict what would happen.

Another key part of Luna’s development: her personality. We wanted Luna to be approachable, optimistic, and a bit sassy. Our target audience was young adults, so the relationship we sought was one between a gym buddy and a coach. The biggest challenge with any bot like Luna is to prevent annoyance — a little extra wittiness gained her much appreciation from our users who often wanting to see what she would say next.

While there are many basic third party dashboards for chatbot analytics, none were able to handle the behavioral information we sought to extract from our users. Accordingly, we built a custom reporting dashboard that allowed us to view transcripts end-to-end, identify individual compliance scores, and see engagement for different parts of Luna’s conversation flows.

In parallel to the conversational design and development, I engaged with our business consultant on the right way to position Luna in the marketplace. Here we were outlining the key elements of the Luna business model based on the many interviews and conversations we had with potential customers.

One of the biggest challenges was having limited resources with large ambitions. We were always prioritizing and re-prioritizing features based on what we we learning from our users and the market.