THE CURRENT STATE OF INTELLIGENT USER INTERFACE

Written by Brooks Canavesi on . Posted in Blog, Mobile App Development, User Experience & Interface Design

The journey of user interfaces is the journey of our interaction with the technology that surrounds us and allows us to do so many wonderful things that wouldn’t be possible just a few years ago. Many people who grew up with computers remember the time when a “green screen” simple command line interface (CLI) was enough to tell the computer everything the user wanted it to do.  Even though I graduated college in 2002, my first job at FedEx Ground had me spending numerous hours of my day in a Korn Shell, Bash Shell, or AS/400 terminal window navigating enterprise class systems and data sets.

As we moved beyond text into the era of graphics, the direct manipulation or WIMP (windows, icons, menus, and pointing) interfaces became the new norm. Their advantage was great discoverability, suitability for multi-tasking, and a certain level of intuitiveness. This same type of user interface has proved to be equally suitable for personal computers, smartphones, and even our home entertainment systems.

However, we are quickly reaching a point where the graphical user interface won’t allow us to take a full advantage of the ever-increasing capabilities of modern artificial intelligent systems backed by big data analysis and a human-like comprehension. We need more effective, efficient, and natural interfaces to support access to information, applications, and people, explains Morgan Kaufmann in Intelligent User Interfaces: An Introduction.

This new type of user interface, the intelligent user interface (IUI), should allow for “the comprehension of possibly imprecise, ambiguous, and/or partial multimodal input; generation of coordinated, cohesive, and coherent multimodal presentations; semi- or fully automated completion of delegated tasks; and management of the interaction (e.g., task completion, tailoring interaction styles, adapting the interface) by representing, reasoning, and exploiting models of the user, domain, task, and context,” explains Kaufmann.

What Are Intelligent User Interfaces (IUIs)

Intelligent User Interfaces: An Introduction defines IUIs as “human-machine interfaces that aim to improve the efficiency, effectiveness, and naturalness of human-machine interaction by representing, reasoning, and acting on models of the user, domain, task, discourse, and media (e.g., graphics, natural language, gesture).”

Naturally, the process of designing and implementing an IUI is a multidisciplinary endeavor, which lies at the intersection of human-computer interaction (HCI / HCD), ergonomics, cognitive science, and artificial intelligence, among other disciplines.

A well-executed IUI must be able to accurately analyze user’s input in one or several of following forms: spoken, typed, or handwritten language; and various gestures. Once the input is processed, the system formulates an output and, optionally, takes into consideration predictions how the interaction could continue, just like we humans do without even thinking about it.

The earliest examples of true IUIs follow the publication of the famous Turing test, developed by Alan Turing in 1950 to test a machine’s ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human. Many subsequent intelligent tutoring systems, such as PLATO (Programmed Logic for Automatic Teaching Operations) were developed for educational purposes and used by universities and government institutions.

While these systems are virtually unknown by the general public, there is one example of IUI that’s deeply engraved into the memory of everyone who is, at least, 20 years old. It is the infamous Office Assistant from Microsoft Office 2000/XP/2003, also known as Clippy. The paperclip was designed to give context-appropriate suggestions and tips to users based on their current actions. It was ultimately disabled in Office XP due to its unpopularity and completely removed in Office 2007.

IUI Versus CUI

With the rise of products such as Amazon Echo, Apple Siri, Microsoft Cortana and Google Now, we have gained the ability to accomplish many everyday tasks just by simply talking with our devices. In his article for Wired.com, Ron Kaplan calls this new user interface paradigm “the conversational user interface”.

He goes on to say, “This is the interface of the future, made even more necessary as computing propagates beyond laptops, tablets and smartphones to cars, thermostats, home appliances and now even watches … and glasses.” Ron continues by stating that “It’s “intelligent” because it combines these voice technologies with a natural-language understanding of the intention behind those spoken words, not just recognizing the words as a text transcription. The rest of the intelligence comes from contextual awareness (who said what, when and where), perceptive listening (automatically waking up when you speak) and artificial intelligence reasoning.”

For all practical purposes, CUI is a sub-category of IUI. We will see very shortly that it definitely isn’t the only one.

The Road Ahead

We still have a long road ahead of us before IUIs become widely used for common applications. Kristina Höök charted how this journey will look like in her paper titled Steps to take before Intelligent User Interfaces become real:

“Unfortunately, there are a number of problems not yet solved that prevent us from creating good intelligent user interface applications: there is a need for methods for how to develop them; there are demands on better usability principles for them; we need a better understanding of the possible ways the interface can utilize intelligence to improve the interaction; and finally, we need to design better tools that will enable an intelligent system to survive the life-cycle of a system (including updates of the database, system support, etc.).”  Numerous Natural Language Processing (NLP) vendors claim to have an intelligent product, however we are still years away from a comprehensive solution to be regularly available.

Despite the numerous challenges, substantial advancements are being made every year by researchers, tech pioneers, and enthusiasts from all over the world. The next big thing for the IUI is ACM IUI 2017, which is the 22nd annual meeting of the intelligent user interfaces community. This time, the event will take place in the St. Raphael Resort in Limassol, Cyprus.

Conclusion

There is no doubt that intelligent user interfaces will play an essential role in the way we interact with the technology around us during the next decade. Entirely new professions will emerge to meet the market demand and push the boundaries of what we deem possible.

BEACON TECHNOLOGY AND MOBILE MARKETING

Written by Brooks Canavesi on . Posted in Blog, Mobile App Development, Software & App Sales, Uncategorized

If you live in a first-world country, chances are that most of your daily activity takes place indoors. Consequently, it might not be possible to use GPS to get accurate locational information. Beacons are a low-cost piece of hardware powered by Bluetooth Low Energy (BLE). Their main purpose is to provide an inexpensive way how to accurately target individual smartphone or tablet users and send messages or prompts directly to their devices.

Even though they are still in their infancy, ABI Research estimates suggest 3.9 million BLE beacons shipped globally in 2015. That’s because retailers, manufacturers, hotels, educational institutions, and governments see how transformative they could be for logistics, customer engagement, and information transmission.

Companies like Zebra are leading the way with innovative products like MPACT.  Zebra’s marketing site states “MPact is the only indoor locationing platform to unify Wi-Fi and Bluetooth® Smart technology, improving locationing accuracy, while allowing you to connect to the most possible customers and capture more analytics and insight. Service is re-defined through impactful interactions with customers via the one device they almost always have in hand – their mobile phone. The result? Instant visibility into where customers are in your facility – and the ability to automatically take the best action to best serve each customer at any time during their visit.”

According to ZDNet, the largest retail deployment of beacons to date was carried out by drug store chain Rite Aid. The company recently announced a distribution of proximity beacons in each of its 4,500 U.S. stores.

Statistics from Swirl, Mobile Presence Management and Marketing Platform, explain why: Relevant mobile offers delivered to smartphones while shopping in a store would significantly influence likelihood to make a purchase for 72% of consumers. What’s more, 80% of consumers would welcome the option to use a mobile app while shopping in a store if that app delivered relevant sales and promotional notifications. That’s a staggering improvement when compared to traditional push notifications, which are opened only about 14 percent of the time, according to mobile advertising firm Beintoo.

As more retailers implement beacons to offer flash sales, provide customers with more product information, and speed up the checkout process, we can expect a dramatic rise in the rate of their adoption. A report from BI Intelligence says that “US in-store retail sales influenced by beacon-triggered messages will see a nearly tenfold increase between 2015 and 2016, from $4.1 billion to $44.4 billion.”

Mobile marketers and developers will have to learn new tricks to fully capitalize on the wealth of opportunities that the beacon technology presents.

TOP 10 OPEN SOURCE TOOLS FOR IOT APPLICATION DEVELOPMENT

Written by Brooks Canavesi on . Posted in Blog, Mobile App Development, Technology Tips & Tricks, Technology trends

Predictions from Gartner paint a clear picture of a future that relies on countless interconnected smart devices just as much as we do on computers and smartphones. According to their predictions, there will be 26 billion Internet of Things devices installed in 2020, generating $300 billion in revenue for manufacturers and service providers and making a $1.9 trillion impact on the global economy.

This article gives an overview of top 10 open source tools for IoT application development. These tools represent a great entry point into this exciting field, which hides a tremendous amount of opportunities for those who are not afraid to learn new things and challenge the current way of life.

1. Arduino Ethernet Shield

The popular open-source electronic prototyping platform used in conjunction with Arduino Ethernet Shield is the perfect hardware combination for simple IoT projects and even more sophisticated applications. All that users have to do in order to control their creations from anywhere in the world is to connect the Arduino board to the Internet with an RJ45 cable and complete a basic setup procedure.

Compared to some heavyweight representatives of the IoT-ready single-board computers, such as the Raspberry Pi or BeagleBone, Arduino is very inexpensive, available virtually anywhere in the world, and used by thousands of enthusiastic users, who love to share their creations with others. Documentation is plentiful and so is the inspiration to turn this wonderful device into something of a great use.

2. Eclipse IoT Project

Eclipse IoT strives to simplify IoT development by implementing IoT standards like MQTT, CoAP, LWM2M, and oneM2M. MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol and the base for the Eclipse Paho project, which provides open-source client implementations of MQTT and MQTT-SN messaging protocols aimed at new, existing, and emerging applications for M2M and IoT.

CoAP stands for The Constrained Application Protocol, and it is a specialized web transfer protocol for use with constrained nodes and constrained networks in the Internet of Things. Lightweight M2M (LWM2M) and oneM2M were created to ensure the most efficient deployment of M2M communications systems.

Together, these standards and tools represent a comprehensive toolkit for any Java programmer who would like to branch out from his or her daily routine.

3. OpenHAB

This vendor- and hardware-neutral open source automation software is designed to let smart devices talk to one another and allow users to add new features to them. openHAB is developed in Java, which makes it possible to run it on any device that is capable of running a JVM. It comes with different web-based UIs as well as native UIs for iOS and Android, and provides APIs for being integrated into other systems.

Creators of OpenHAB recognized that proprietary smart devices quickly become obsolete, incompatible, and unable to meet security requirements of modern-day users. Their solution leverages data from all available subsystems and integrates them into one cohesive package. The tool received a People’s Choice Winner at the Postscapes IoT Awards 2014/15 and the Duke’s Choice Award 2013.

4. RIOT OS

RIOT is an open-source operating system developed by a grassroots community to power various Internet of Things solution. When compared to regular Linux or other IoT operating systems, such as Tiny OS and Contiki, RIOT supports both C and C++, comes with multi-threading with ultra-low threading overhead (<25 bytes per thread), real-time capability due to ultra-low interrupt latency (~50 clock cycles) and priority-based scheduling, and excellent modularity.

There’s even a native port of RIOT that allows developers who are not familiar with embedded programming to run RIOT inside a process on Linux or Mac OS. All code is hosted on GitHub and the provided wiki contains plenty of useful information to get started with RIOT.

5. Thinger.io

Thinger.io provides a ready to go scalable cloud infrastructure that can be controlled with their easy to use admin console, or integrated into a business logic with REST API. The entire project is open-source and completely hardware agnostic. The ready-to-use scalable cloud infrastructure allows for easy deployment of your own infrastructure, both in local machines or in the cloud.

As such, Thinger.io essentially eliminates the need to select a compatible vendor hardware and use bloated software to accomplish very simple things.

6. OpenIoT

OpenIoT is a joint effort of prominent open source contributors who want to provide support for cloud-based and utility-based sensing services. This middleware will support flexible configuration and deployment of algorithms for collection, and filtering information streams stemming from the internet-connected objects, while at the same time generating and processing important business/applications events, according to OpenIoT’s GitHub page.

The entire architecture consists of three main planes: the Utility/Application Plane, the Virtualized Plane and the Physical Plane. Each plane includes several elements, such as the Request Definition, Request Presentation and Configuration and Monitoring components. OpenIoT comes with a built-in Scheduler, Cloud Data Storage, and even Service Delivery & Utility Manager.

7. IoTSyS

OpenIoT is not the only integration middleware for the Internet of Things in existence; there is also IoTSyS, which provides a communication stack for embedded devices based on IPv6, Web services and oBIX to provide interoperable interfaces for smart objects. The main objective of IoTSyS is to leverage the power of existing automation systems and sensors and use it easily create and deploy a new solution, while addressing security, discovery, and scalability issues.

8. Freeboard

Not to be mistaken with the unique skateboard, Freeboard is an elegant dashboard for the IoT. It allows anyone to quickly build real-time, interactive dashboards and visualizations using the intuitive drag & drop interface. The dashboard stands on top of a secure, high-performance, enterprise-class cloud system and the entire project is open-source and publically hosted on GitHub.

It features seamless integration with dweet.io, or access any web-based API, the ability to select from a growing list of included widgets, and instant sharing via email, SMS, and social networks. Some notable examples include The Heising-330, which is a modern, internet-connected, and fully automated continuous still, built by and for the craft distillery and a prototype of air quality monitoring dashboard.

9. Interstacks

“Interstacks are snap-together electronic blocks and Stackbuilder visual authoring tool. In minutes, invent any smart devices you can imagine. Then connect them to each other and the internet. Interstacks empowers you to become master of your internet of things universe, according to the official website.

Getting started with Interstacks is made simple thanks to the modular approach and desktop applications for Windows and Mac built around the Phyton programming language. The holistic approach to prototyping allows users to configure their system of devices and internet services, build macros, and write rules to automate systems.  Maya Design created Interstacks & Stackbuilder products with decades of user interface / user experience (UI/UX – HCD) expertise guiding their innovative and intuitive approach.   The Stackbuilder interface is a visual authoring tool (drag-and-drop) that enables the user to build, test and tweak their stacks in a very intuitive fashion.  The concept of rapid prototyping lives at the core of this product line.

10. Zetta

Built on Node.js, Zetta is an open source platform for creating Internet of Things servers that run across geo-distributed computers and the cloud. Zetta achieves this by combining REST APIs, WebSockets and reactive programming.

Once installed, Zetta servers can run everywhere and have no problems communicating with Arduino, Spark Core, and other microcontrollers. Coding is simplified by a series of helpful abstractions that allow developers to focus on the big picture and not get too bogged down with insignificant details.

Conclusion

Even though these 10 tools cover everything from application development to middleware and home automation suite, we have barely scratched the surface of what’s currently available. The Internet of Things is going to be one of the several key technologies of the future and the knowledge of its common building blocks (no pun intended “interstacks”) is likely to prove extremely useful.

SMART CLOTHING: THE NEXT WAVE OF WEARABLE TECHNOLOGY

Written by Brooks Canavesi on . Posted in Blog, Technology trends

It seems that we have finally reached the point where wearable device manufacturers are able to look beyond the fitness industry and start coming up with interesting ideas for the next wave of wearable technology that would be equally useful for those who spend most of their days in business attire and even improve the overall accuracy of measurements.

Chris Harrison, an assistant professor of human-computer interaction who leads the Future Interfaces Group at Carnegie Mellon University (Pittsburgh, PA) explains that “You can make millions of smart watches that are identical, but you have millions of people who are not identical.” The individual variations in the thickness of our wrists, the amount of fat we store in the area, or even how much we sweat can profoundly influence the accuracy of measurements. Furthermore, wrists are exposed to plenty of abuse throughout the day, making them even less suitable as the place for an expensive, high-end gadget.

OMSignal, Hexo Skin, Samsung, Google, and Under Armour have introduced smart clothes that abandon wrists in favor of less conspicuous body parts. OMbra by OMsignal uses a cutting-edge biometric fabric and integrated sensors to measure user’s heart rate, balance, breathing, stamina levels, distance traveled, calories burned, and plenty more. A similar product aimed at men is called Enflux. Instead of just a sports bra, Enflux Exercise Clothing consists of “a compression shirt and pants with ten small embedded motion sensors,” according to the Kickstarter campaign. Because the suit can capture all body parts at once, athletes can replay their performance using 3D animations with useful metrics, which include the precise angle and velocity of the body parts. Other highly similar fitness-oriented products include the MyZone Sports Bra, the Lumo Run shorts, the LikeAGlove leggings, the Athos shirt and shorts, or the sports bra from Victoria’s Secret.

This new wave of smart fitness products is powered by several innovative technologies, which “allow for the ‘detection, transmission and protection of electrical signals’ within smart clothing,” explains General Manager of Noble Biomaterials, Bennett Fisher, who added that “Once the sensor is inside the clothing, what you’re wearing becomes a sensor.” A similar technology was presented during Google I/O 2015 by the founder of Google Project Jacquard, Ivan Poupyrev. In their own words, the technology makes it possible to weave touch and gesture interactivity into any textile using standard, industrial looms. That means that everyday objects such as clothes and furniture can be transformed into interactive surfaces. They achieve this by intertwining a conductive metal core with conventional fibers that can be dyed any color. Unsurprisingly, some of the biggest clothes manufacturers in the world, including Levi’s, are already exploring way how they could use this technology to create something their customers would love.

The current selection of everyday smart clothes revolves mostly around the NFC technology. A great example is a smart suit from Samsung. It was built in collaboration with Rogatis. Those who are willing to spend around $500 on it can enjoy the ability to unlock their phone, or digitally swap business cards. The Lyle & Scott contactless jacket throws in a contactless payment chip found in credit cards to make contactless payments even more convenient.

So, how long it could take to see the technology used outside of the fitness industry? Quite a long time, actually. “Building these products [health-oriented smart devices] takes lots of time. Testing, simulations, modeling, prototyping, and problem-solving are all more extensive when you need to make sure the devices can stand up to the requirements of daily wear, such as frequent exposure to sweat and water”, explains Rachel Metz in her article for Technology Review.

That being said, the competition to be the first to dominate the market is fierce. VTT Technical Research Centre of Finland has developed a promising technology that allows smart clothes or adjusts the temperature based on wearer’s individual needs. Such technology has a lot of potential for the health care industry, police officers, firemen, outdoor workers, and even small babies, who already are the target audience for products such as Exmobaby, Owlet Baby Care, and MonBaby from MonDevices.

The first mentioned is like a baby monitor on steroids. This smart garment can tell whether the baby is sleeping, check the vital signs, and keep the parents up to date with its Wi-Fi and 3G wireless capability. The other two products follow a very similar formula, each with its own distinct set of features and overall value propositions.

With all these innovations taking place all around us, it’s easy to see how the future of smart clothing is going to look like: it’s going to be more practical, less fitness-oriented, and infinitely more embraced by customers of all walks of life.

Fitness Trackers: A Way to Good Health or a Big Privacy Threat

Written by Brooks Canavesi on . Posted in Blog, Mobile App Development, Technology trends

Since their first appearance on the consumer market, people from all around the world have fallen in love with fitness trackers. Data from Statistica reveal that there were more than 13 million sold in the United States in just the last two years. These devices are great at keeping people motivated to adhere to healthy exercise habits, monitor their daily caloric intake, or watch out for dangerously high stress levels. But many security experts and technology analysts are unsure whether they are equally great at protecting all the associated private data.

A typical fitness tracker (also known as activity tracker) is an electronic device fitted with a wide assortment of specialized sensors that measure anything from heart rate to the quality of sleep, distance walked, body and ambient temperature, elevation, acceleration, speed, position, or calorie consumption. We can expect that future fitness trackers will be FDA-approved devices capable of alerting users to medical problems and suggesting the best possible remedy or action to take.

Integrated sensors collect various information, which are then both processed locally to display results on the device itself and sent to the cloud in order to allow for multi-platform access and management. This is exactly where the biggest problem is: most users are not aware of how their data are being used, who exactly can access them, and how they could be used for identity theft in case it falls into the wrong hands.

This is not surprising at all. With the average terms of service agreement nearing 5,000 words, the temptation to skip all that hassle and just click on the “agree” button is very high. However, when customers do so, they unknowingly give access to their private data to third-party companies, as discovered by the Federal Trade Commission (FTC) in 2014. The commission reported that a sample of a dozen health and fitness apps collectively sent data to 76 third parties. What exactly can those third-party companies do with users’ data is only between them and the manufacturer of the particular fitness tracking device.

To outline some of the contemporary data handling issues in this industry segment, Open Effect, a Canadian not-for-profit that conducts research and advocacy efforts focused on ensuring people’s personal data is treated securely and accountably, create a report called Every Step You Take.

They studied some of the most popular fitness tracking applications in the Google Play store as of mid-2015 and asked the following questions: What technical security mechanisms are in place? How could they be exploited? What categories of data does each device actually collect?

All device except for one transmitted their data over the internet. In some cases, this included sensitive and unnecessary information such as the IMEI number or fine-grained location data. Out of all devices, only the Apple Watch randomized the MAC address as a protective measure against persistent monitoring of the wearer’s presence. What’s much more alarming than the absence of the MAC address randomization is Garmin’s failure to implement HTTPS encryption to secure the transmission of personal information. The company has since then corrected this flaw, but nobody can know for sure how many other devices on the market remain similarly defenseless against even the most basic types of attacks.

As explained by Theresa Payton, president and CEO of Fortalice and a former White House CIO, “The culprit is the innovation life cycle. There is tremendous pressure to get cool and affordable products on the market at a dizzying speed.” She went on to say that wearables and associated apps “have a track record of poor privacy and security measures.”

A good news is that substantial effort has been and is being made to protect the privacy and security of end-users. The IEEE Center for Secure Design released a paper titled, “WearFit: Security Design Analysis of a Wearable Fitness Tracker,” to “addresses each of the top 10 software security design flaws” of fitness trackers and show developers of these devices how to design a product that meets all modern security standards.

Their WearFit system is an imaginary wearable personal health monitoring device that resembles many currently available products. The device can measure step count and heart rate and sends the data to a compatible mobile application, which then communicates with the platform backend. Many common attack vectors were taken into consideration, including Denial of Service (DoS), falsifying the users’ data, stealing users’ data via SQL injection or phishing, and, for example, compromising device integrity with malicious firmware updates.

It’s hard to predict what future holds for fitness trackers. As customers are becoming increasingly more conscious of their privacy and security, companies will have to ensure that their products are free of any vulnerabilities that could lead to a data leakage or a loss of sensitive information. Customers themselves should demand a high level of transparency when it comes to how exactly their data are handled and used. Only then we’ll be able to embrace all latest technological innovations and used them to improve our daily lives.