Qt World Summit Online 2020 Presentation

1. Lessons Learned
from Building 100+
C++/Qt/QML Devices
Peter Winston,CEO, Integrated Computer Solutions

2. Peter Winston, Chief Executive Officer
Peter founded Integrated Computer
Solutions, Inc. (ICS)in 1987. Through his
leadership and innovation, the company
has experienced continued growth,
expanded across the globe and today is
focused on helping clients design and
deliver powerful applications on
touchscreen and embedded technology.
About ICS

3. About ICS
> Inventing the future
• ICS helps companies design, develop, and
productize touch, gesture and voice-enabled
solutionsthat dramatically improve customer
experience
> Qt Software Development Services & Training
• Our average developer has 15 years of Qt experience
> Full Suite of User Experience (UX) Design Services
• Design studio, Boston UX

4. Our Markets
Automotive
Industrial Consumer
Aerospace & DefenseMedical & Life Sciences
Commercial Kitchen

5. A Few of Our Customers

6. Why Projects Fail
And how to set them up to succeed

7. Unknowns

9. Why It’s Hard to Build a Modern Device

10. Elements of Modern Devices
> Graphic design
> Standard controls
> Interactive controls
> Animation
> Real Time
> Complex device control
> Sensors and actuators

11. Elements of Modern Devices
> Settings
> Wifi, Bluetooth, NFC, GPS
> On screen keyboard
> Charts
> Video
> Voice
> Sound

12. Elements of Modern Devices
> Administrative console
> Settings
> Alarms
> Error logging
> Security
> Over the air updates
> Usage analytics

13. Elements of Modern Devices
> Websockets- web pages, PDFs
> 2nd screen/mobile/web interface
> Indoor location sensors
> Maps and navigation
> Internal monitoring

14. Elements of Modern Devices
> Users- who expect a great interface
> And things should just work
> Back
> Undo
> Preferences

15. While….
> Running great on cheap hardware

16. Why It’s Hard to Build a Modern Device

17. Project Principles
> Find risk early
> Use UX to test and nail down features
> Build simple prototypes for most difficult parts
> Use experienced staff on each component
> Test throughout
> Reuse software wherever possible

18. Qt UI Principles
> Separate UI from backend of application
• Never put application logic into QML component
• Keep flexibility between layers
> More C++ / Less QML
> Let UX lead, and implement the details
• Pixel perfect, with sound, motion, etc.

19. Layered Architecture for Qt HMI’s
Name Me
Name Me
Name Me

20. Layered Software
Device Control and I/O
User interface (C++)
GUI presentation (QML)
Platform & device drivers
Messaging
Framework
&
Rules engine Data
Application
logic
call()
Signal()
> Make layers testable
> State machine for navigation and event
dispatch — flexible yet testable
> Make components reusable

21. Device Architecture
Device Control and I/O
User interface (C++)
GUI presentation (QML)
Platform & device drivers
Messaging
Framework
&
Rules engine Data
Application
logic
Device Control
Display Screen

22. Device Architecture
Device Control
Display Screen
Device Control and I/O
User interface (C++)
GUI presentation (QML)
Platform & device drivers
Messaging
Framework
&
Rules engine Data
Application
logic

23. Device Architecture
Device Control and I/O
User interface (C++)
GUI presentation (QML)
Platform & device drivers
Messaging
Framework
&
Rules engine Data
Application
logic
Display Screen
Over-the-Air Updates
Analytics
Commerce
Support
Data Storage
Voice Control
Mobile interface
Secure
Access
Device Control

24. Device Architecture
Device Control and I/O
User interface (C++)
GUI presentation (QML)
Platform & device drivers
Messaging
Framework
&
Rules engine Data
Application
logic
Device Control
Display Screen
Over-the-Air Updates
Analytics
Commerce
Support
Data Storage
Voice Control
Mobile interface
Secure
Access

25. Device Architecture
Device Control and I/O
User interface (C++)
GUI presentation (QML)
Platform & device drivers
Messaging
Framework
&
Rules engine Data
Application
logic
Display Screen
Over-the-Air Updates
Analytics
Commerce
Support
Data Storage
Voice Control
Mobile interface
Secure
Access
Device Control

26. Example Project
> Is there a typical HMI?

27. Open Source Ventilator

28. Main Screen

29. Platform and Dev Environment
> Platform
> Raspberry Pi 4- 2 Gig Ram
> 10” touchscreen
> OS- up to ICS (Raspbian or Yocto)
> Toolkit - up to ICS Qt/QML (OpenGL?)

30. Electrical Design
Custom RespiraWorks PCB
Blower
Display +
Alarms
Fuse
12V IN
Pressure
Flow (dP)
Flow (dP)
5V Supply
Humidifier
Sensing
tubes
Flashing
Light/
Alarm
Blower Driver
12V Power
5V Power
Signal
Voltage Monitor
FILTERING
Outlet
Solenoid
Heater Switch
Solenoid Switch
STM-32
Raspberry Pi

31. > Identify requirements
UX Design

32. Wireframes

33. Wireframes Continued
7/23/2020

34. Wireframes Continued
7/23/2020

35. Choice of Themes
7/30/2020

36. ProjectRed UI
8/1/2020

37. 8/1/2020

38. Flow

39. UX Process
Requirements
User Stories
Wireframes
Graphic design
Flow

40. UX→ UI
Requirements
User Stories
Wireframes
Graphic design
Flow
User interface
Navigation

41. The UI Simulator
> Test UX and UI

42. ICS UI Simulator Architecture
Hardware Software
Update
Display User interface (C++)
GUI presentation (QML)
Non-target hardware
Web/assembly
Framework to support
navigation, respirator state,
and sample data
{
UI Simulator

43. UI Simulator
Allows control and display of all parts of the UI without a working device

44. UI Simulator Results

45. Alarm Settings- Old vs New
6/15/2020 8/13/2020

46. Alarm Settings Redux

47. Alarm Setting Detail

48. Ventilator Settings - Drawer
Collapsed Expanded

49. The Wave Form
> Identify and move risk forward

50. The Wave Form- Detail
ProjectRed main screen 8/13/2020
> How fast?
> Real time?
> Overwrite or scroll?
> Signal processing?
> QML, OpenGL, or GPU
odd

51. Recorded Prototype Waveforms

52. > Plan
> Start early
> Test on target HW

53. Application Logic

54. Workflow/State Machine

55. Communication

56. Message Driven GUI
> Insulates UI from changes made to MCU interface
• Using“Google Protocol Buffers”framework
- Send messages to MCU to update parameters
- Query MCU for response and update GUI accordingly

57. Integrated Computer Solutions Inc. www.ics.com
Data connector
Connects QSerialPort data with Qt properties
int tidalVolume
int Peep
int pPeak
int mVI
int Rate
int vti
Qt Signals trigger
screen updated
Data Connector
MCU
Serial data Update every 30ms
everys

58. Integrated Computer Solutions Inc. www.ics.com
UI - Backend Interface
“GUIMessages” object methods
getConfigurationCmd (query MCU for GUI configuration file)
startTestCmd (request MCU to start test procedure)
getTestStatusCmd (query MCU for tests results)
runFunctionCmd (request MCU to run a function)

59. Device Simulator
> Unit test the HMI

60. Plugins Based Backend
> Backend split into two plugins:
• Production code backend that connects to MCU hardware
• Demo backend for development powered by simulator
Allows for fast and easy switch between development and
production code. Share the sameinterface.

61. Device Simulator
Device simulator stands in for
the MCU, sending and
receiving simulated data
Enables testing of edge cases
and error conditions
In this case it was playback of
recorded data
Testing communications API,
waveform and alerts,
conditions

62. ICS Device Architecture
Hardware Software
Update
Display
Trigger
Action
Device Control and I/O
Application
logic
User interface (C++)
GUI presentation (QML)
Non-target platform
Data
Rules
engine &
Framework
Device Simulator
Replaces hardware I/O

63. Integration and Testing

64. ICS Advantage
> Using layers forces each layer to be independent of the others, which in
turn lets us work in parallel to bring the schedule in.
> Doing many projects this way creates a standard process, toolchain and a
framework which facilities the layers and reusable components.
> This in turn delivers more higher quality projects and enables us to surge
on project when required.

65. Recap
> Identify risk early
> Use UX to define requirements
> Build a UI prototype and test it with customers
> Build in layers and build up to the application
> Get the hard part working early
> Create components that are both testable and reusable
> Be prepared to surge, or to limit scope

66. How Can We Help You?
Deliver the best Qt applications for touch, embedded and
desktop with ICS
> New Qt projects benefit from our proven process
> Existing projects finish faster with our Qt expertise
www.ics.com• www.bostonux.com

67. Questions or Comments
> peter.winston@ics.com
> https://www.linkedin.com/in/peterwinston/