AimBot: Integrating Assistive and Informative Haptics through Field-Oriented Controlled (FOC) Motor Force Dynamics
Abstract: In this research, I present the design and evaluation of AimBot, a 2-axis input device with a ball driven by two Field-Oriented Controlled (FOC) BLDC motors that moves in X and Y axis and also manipulated in a similar way to the trackball mouse. The two driving forces enable what I defined as Assistive Haptics: the motor end transfers not only directional information but also directional actuation to the hand.
The research adapts the concept of Input-Output-Coincidence, which associates the input to the output, the sensor to actuator. Closed-loop FOC, an important control method for mechatronics and robotics, is an appropriate technical solution for this purpose. As an alternative to conventional Human Interface Device (HID), AimBot is an input device that can be attached to various types of traditional UI to realize features proposed in Tangible UI (TUI). The concept of Force Image Schema (FIS) is also used and elaborated in the research to evaluate the signifying process of the haptics.
I use a research-by-design approach that explores the possibilities of FOC-based haptics for its capabilities to dynamically fine-tune the feedback in real time, which enables informative and assistive interactions with the potential to be implemented in many HCI scenarios such as non-visual and high-performance control.
“True, this sort of sensation is rather confused and obscure in those who do not have much practice with it; but consider it in those who, being born blind, have made use of it all their lives, and you will find it so perfect and so exact that one might almost say that they see with their hands, or that their stick is the organ of some sixth sense given to them in place of sight.”
René Descartes, Dioptrique
Sources
Capstone Project
Winter 2024
#HCI #Haptics
#Mechatronics
#Tangible #Assistive
Introduction: Rethinking Haptic Interface
Mapping the physical to the digital
In the history of UI for computational systems, the HID (Human Interface Devices) of some kind is always present, while the form-factor turned out to be a limiting factor as the UI expands beyond GUI. This one and only physical device that’s directly touched and manipulated by user’s body is always overlooked.
Input-Output Coincidence
The concept of Input-Out Coincidence is an concept concerning the physical integration of sensor and actuator. Being proposed by the researchers of Tangible UI, the term is never as popular as the TUI itself. Tracing down the history of UI, I think the principles of TUI shall be incorporated for the advancement of the design of the interface devices.
“Transform its shape to reflect underlying computational state and user input (Shape Change);
Conform to constraints imposed by the environment and user input (Dynamic Constraints);
Inform users of its transformational capabilities (Dynamic affordances)”
“The vision argues that with physical embodiment a strong input and output coincidence we can leverage the physical dynamic affordance real world objects give us and our ability to assume how an interface will behave in response to manipulation through exploration of those affordances.”
Luke Vink, Materiality in Suspense
Scratching in Turntablism
A coincidence in the musical interface which can exemplify the idea of haptics in input-output coincidence. DJ’s scratching technique is referring to manual input that disrupts the music output through direct (and analog) manipulation of counterbalancing the motor drive of the vinyl with the hand drive.
This graph presents my own development of the I/O Coincidence concept, giving possibility to a new dimension of interactivity that’s more than omni-directional control of input device but assistance, a form of output with a more effective and intuitive signification process.
Past Works: Haptic Input-Output
Within the concept of I/O Coincidence, a technical path shall be determined. Haptics is, throughout the history of input device, the kind of technology that is rarely implemented with success yet it’s important as it opens one of the most important sensation of human body. Potentially, a more dynamic and sense-making type of haptic device can be designed with the concept of I/O Coincidence. There are existing examples of research or product that touches on this possibility.
Case Study: SmartKnob
In 2020, a project named "SmartKnob" went viral on Youtube, the smartness of it appears in the software-defined dynamic feedback and detent. The knob’s rotation behavior can be changed to different patterns: continuous smooth rotation; high-resolution knob; ratchet with various pitch; spring-loaded rotation that always returns to the start; bistable switch, etc..
Field-Oriented Control (FOC), the core technology used in the SmartKnob’s servo system. It is always used as a closed-loop control method optimized for low-speed, precise motor control. The version of it popularized by the SmartKnob includes a magnetic encoder and radial magnet attached to the shaft of the BLDC motor. As a technical solution, the closed-loop control system like FOC echoes with the concept of I/O Coincidence as its connecting input and output into a loop, which is a fundamental concept in automation and cybernetics.
FIS(Force Image Schema)
“The idea of a special group of force image schemas originates in cognitive linguistics. Force image schemas are cognitive representations of our naïve understanding of physical force dynamic events in the world.”
Hurtienne et al., Comparing Pictorial and Tangible Notations of Force Image Schemas
FIS can be used both physically and metaphorically in tangible interaction design. They can represent real physical forces or express abstract information through haptic feedback.
Methodology: design and evaluate
The design process starts with the validation of the interactivity and design the mechanics that enables the form of control discussed in the research phase. The form-factor presented in TouchBall (Cho and Kim, 2009) is worth noting as it adopts the structure of a trackball mouse as its basic form. Trackball is possibly the only controller that can achieve infinite, omnidirectional movement, it is a potential path for our project.I also encountered two retro tech that hint the possible mechanical solution to a two-way driving controller/actuator: analog trackball and mechanical mouse. They are both translating the omnidirectional ball movement with two rollers attached to encoders on two axes.
Functional Validation:
A 16:9 X-Y traditional gantry motion system powered by FOC controlled BLDC motor. It is made to validate the servo system, test motors, and validate the proposed haptic-assisted navigation.
Mechanical Test: Friction
The friction-driven mechanics presents a key challenge in optimizing three friction coefficients: Hand-to-Ball (HtB), Ball-to-Housing (BtH), and Ball-to-Roller (BtR). Notably, BtH, which mediates movement transmission, must be minimized rather than maximized for optimal performance. Multiple types of materials for all three parts were tested. The outcome of the testing is reflected on the design choices. On the housing, three ball transfer bearings are used to minimize BtH, and two other parts used polyurethane rubber to maximize HtB and BtR.
Prototype v1.0
The first prototype is the outcome of the friction test, it is a test bench built for further software testing. It includes all the findings in the mechanical tests and enabled me to iterate on the dimensions.
Prototype v2.0
The second prototype is one step further to integrate all parts into a more finished enclosure. I optimized the ergonomics by adding a waist rest and tilting the enclosure for the mechanics (motors and ball) 10 degrees downward so that it would not block the hand movement. The inside of the enclosure is used to store electronics. Rubber feet are added for stability.
Mechatronics
The electronics used in the project are simple and I chose them to be all widely available open hardwares. I especially chose the M5stack RollerCAN as the FOC motor module in the second iteration as it's an open hardware with standardized specs.
Control
The control system is, of course, a closed loop in which I used CAN bus to control the motor and read encoder data. A series of responsive, dynamic haptics can be achieved by comparing three position inputs in the system, mediated by PID controller.
Results: Haptic-Assisted Control
FOC for FIS
One FIS notation can simulate one tangible, physical phenomenon. With AimBot’s fine-tuning of behavior of the two motors, it is possible to actuate the pictorial notations of FIS as sequences of directional force patterns: if a single tangible notation of FIS is a word, AimBot is able to speak.
Desktop Navigation
Derived from the FIS of Attraction, this is an application in assistive technology which enables a non-visual, actuated navigation on 2-axis desktop/web UI.With in the system, there are three values send from computer to the AimBot: Force, ErrorX, ErrorY.
Works Cited
Choi, Minwoo, and Gerard Jounghyun Kim. “TouchBall: a design and evaluation of a hand-held trackball based touch-haptic interface.” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 2009, pp. 1535–1538. Association for Computing Machinery, New York, NY, USA, https://doi.org/10.1145/1518701.1518936.
Derrida, Jacques. On Touching-Jean-luc Nancy. Stanford University Press, 2005.
Descartes, René. Discourse on Method, Optics, Geometry, and Meteorology. Hackett Pub., 2001.
Lo, Jo-Yu, et al. “RollingStone: Using Single Slip Taxel for Enhancing Active Finger Exploration with a Virtual Reality Controller.” Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18), 2018, pp. 839–851. Association for Computing Machinery, New York, NY, USA, https://doi.org/10.1145/3242587.3242627.
“Mouse Alternatives for Disabled Users - Assistive Technology.” Pretorian Technologies, https://www.pretorianuk.com/mouse-alternatives. Accessed 26 November 2024.
Parisi, David. Archaeologies of Touch: Interfacing with Haptics from Electricity to Computing. University of Minnesota Press, 2018.
Random Access Projects. “Arduino Brushless Motor DIY Haptic Interface.” YouTube, 30 October 2020, https://www.youtube.com/watch?v=9Eh1p_rUQMA. Accessed 26 November 2024.
scottbez1. “DIY haptic input knob: BLDC motor + round LCD.” Youtube, 11 March 2022, https://www.youtube.com/watch?v=ip641WmY4pA. Accessed 26 November 2024.
Stiegler, Bernard. Technics and Time: The fault of Epimetheus. Stanford University Press, 1998.
Wobbrock, Jacob, and Brad Myers. “Trackball text entry for people with motor impairments.” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 2006, pp. 479–488. Association for Computing Machinery, New York, NY, USA, https://doi.org/10.1145/1124772.1124845.
Xie, Mufeng, et al. “A 3-D Haptic Trackball Interface for Teleoperating Continuum Robots.” 2022 9th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), 2022. IEEE, https://ieeexplore.ieee.org/document/9925384/authors#authors.
