Michigan Robotics at IROS 2023 in Detroit

This paper presents the gatekeeper algorithm, a real-time and computationally-lightweight method to ensure that nonlinear systems can operate safely within unknown and dynamic environments despite limited perception. Gatekeeper integrates with existing path planners and feedback controllers by introducing an additional verification step that ensures that proposed trajectories can be executed safely, despite nonlinear dynamics subject to bounded disturbances, input constraints, and partial knowledge of the environment. Our key contribution is that:

1. We propose an algorithm to recursively construct committed trajectories.
2. We prove that tracking the committed trajectory ensures the system is safe for all time into the future.

The method is demonstrated on a complicated firefighting mission in a dynamic environment and is compared against state-of-the-art techniques for similar problems.

Lower-limb exoskeletons may experience errors in operational settings, where an expected assistive torque is missing. These errors may affect the user’s gait strategies and perception of the exoskeleton’s performance, leading to impacted human-exoskeleton fluency and user trust in the system. In this study, we introduced five different exoskeleton control algorithms with fixed error rates up to 10% error (90% accuracy). Two groups of participants (N=12, 6 per group) walked with a bilateral ankle exoskeleton while completing a targeted stepping task and experienced each controller twice, but in different orders. The impact of exoskeleton error rates was assessed on step characteristics (step length and width), task performance (absolute task error), and perception of exoskeleton performance (survey responses). Step characteristics were not impacted by exoskeleton errors, but multiple participants were not able to achieve acceptable task accuracy and increased task error over time across all error rates. Increasing error rates negatively impacted users’ perception of algorithm predictability, exoskeleton supportiveness, and probability of future usage. The threshold where perceived predictability and future usage probability transitioned from positive to negative occurred between 2% and 5% error. Understanding the effect of increasing exoskeleton error rates informs minimum algorithm accuracy to support human-exoskeleton fluency and performance for gait-assist exoskeletons.

Exoskeletons have shown great potential to enhance locomotion by augmenting the lower limb. While most research has focused on steady-state ambulatory activities, the ability to assist transient, ballistic tasks is also important for understanding the role of exoskeletons in mobility enhancement. In this preliminary study (N = 5), we developed an individually-customized control strategy to assist vertical jumping. The control strategy was deployed on bilateral ankle exoskeletons (ExoBoot, Dephy Inc.). We structured the control strategy as a work loop that parameterized the assistance provided during the jump. We show that configuring the controller based on individual biomechanics and user preferences facilitates increased vertical jump height when using exoskeleton assistance. In addition, we demonstrate that a user’s squat depth can have a significant (p < 0.05) impact on height achieved, but that this depth does not need to be optimized; rather, the exoskeleton provides the maximum performance assistance from both preferred- and deep-squat conditions. Jump height increased by 7.2% with the exoskeleton at its maximum assistance setting, which is comparable or greater than previous systems.

Positive biomechanical outcomes have been reported with lower-limb exoskeletons in laboratory settings, but these devices have difficulty delivering appropriate assistance in synchrony with human gait as the task or rate of phase progression change in real-world environments. This paper presents a controller for an ankle exoskeleton that uses a data-driven kinematic model to continuously estimate the phase, phase rate, stride length, and ground incline states during locomotion, which enables the real-time adaptation of torque assistance to match human torques observed in a multi-activity database of 10 able-bodied subjects. We demonstrate in live experiments with a new cohort of 10 able-bodied participants that the controller yields phase estimates comparable to the state of the art, while also estimating task variables with similar accuracy to recent machine learning approaches. The implemented controller successfully adapts its assistance in response to changing phase and task variables, both during controlled treadmill trials (N=10, phase RMSE: 4.8 ± 2.4%) and a real-world stress test with extremely uneven terrain (N=1, phase RMSE: 4.8 ± 2.7%).

Robotic ankle exoskeletons have been shown to reduce human effort during walking. However, existing ankle exoskeleton control approaches are limited in their ability to apply biomimetic torque across diverse tasks outside of the controlled lab environment. Energy shaping control can provide task-invariant assistance without estimating the user’s state, classifying task, or reproducing pre-defined torque trajectories. In previous work, we showed that an optimally task-invariant energy shaping controller implemented on a knee-ankle exoskeleton reduced the effort of certain muscles for a range of tasks. In this paper, we extend this approach to the sensor suite available at the ankle and present its implementation on a commercially-available, bilateral ankle exoskeleton. An experiment with three healthy subjects walking on a circuit and on a treadmill showed that the controller can approximate biomimetic profiles for varying terrains and task transitions without classifying tasks or switching control modes.

Emerging partial-assistance exoskeletons can enhance able-bodied performance and aid people with pathological gait or age-related immobility. However, every person walks differently, which makes it difficult to directly compute partial-assistance torques from joint kinematics. Gait-state estimation-based control allows more consistent behavior between users by using phase (normalized stride time) and task variables (e.g., stride length and ground inclination) to parameterize the joint torques. Using kinematic models that depend on the gait-state, prior work has used extended Kalman filter (EKF) to estimate the gait-state online. However, the EKF also suffers from kinematic errors since it uses a subject-independent measurement model, and it is still unknown how personalization of this measurement model would reduce gait-state tracking error. This paper quantifies how much gait-state tracking improvement a personalized measurement model can have over a subject-independent measurement model when using an EKF-based gait-state estimator. Since the EKF performance depends on the measurement model covariance matrix, we tested on multiple different tuning parameters. Across reasonable values of tuning parameters that resulted in good performance, personalization improved estimation error on average by 8.5 ± 13.8% for phase (mean ± standard deviation), 27.2 ± 8.1% for stride length, and 10.5 ± 13.5% for ground inclination. These findings support the hypothesis that personalization of the measurement model significantly improves gait-state estimation performance in EKF based gait-state tracking (P < 0.05), which could ultimately enable reliable responses to faster human gait changes.

This article presents a new slip estimator for mobile robots that employs the Disturbance Observer (DOB) and the invariant observer design theory. The proposed state estimator is entirely proprioceptive and combines data from an inertial measurement unit and body velocity. It uses a Right Invariant Extended Kalman Filter (RI-EKF) and embeds the slip velocity, together with spatial vectors (velocity and position), into the standard RI-EKF on the Lie group SE(3). By doing so, the slip velocity is represented as an additive Lie group structure, and the entire state is formed on the Lie group SE3(3). Furthermore, this research investigates the observability of the proposed observer and demonstrates that it is fully observable. The DOB-based RI-EKF provides precise, real-time estimates of velocity and slip velocity on various terrains. Experimental results using a Husky wheeled robot confirm the mathematical derivations and show better performance than a standard RI-EKF baseline. Open source software is available for download and reproducing the presented results.

A new control paradigm using angular momentum and foot placement as state variables in the linear inverted pendulum model has expanded the realm of possibilities for the control of bipedal robots. This new paradigm, known as the ALIP model, has shown effectiveness in cases where a robot’s center of mass height can be assumed to be constant or near constant as well as in cases where there are no non-kinematic restrictions on foot placement. Walking up and down stairs violates both of these assumptions, where center of mass height varies significantly within a step and the geometry of the stairs restrict the effectiveness of foot placement. In this paper, we explore a variation of the ALIP model that allows the length of the virtual pendulum formed by the robot’s stance foot and center of mass to follow smooth trajectories during a step. We couple this model with a control strategy constructed from a novel combination of virtual constraint-based control and a model predictive control algorithm to stabilize a stair climbing gait that does not solely rely on foot placement. Simulations on a 20-degree of freedom model of the Cassie biped in the SimMechanics simulation environment show that the controller is able to achieve periodic gait.

For bipedal humanoid robots to successfully operate in the real world, they must be competent at simultaneously executing multiple motion tasks while reacting to unforeseen external disturbances in real-time. We propose Kinodynamic Fabrics as an approach for the specification, solution, and simultaneous execution of multiple motion tasks in real-time while being reactive to dynamism in the environment. Kinodynamic Fabrics allows for the specification of prioritized motion tasks as forced spectral semi-sprays and solves for desired robot joint accelerations at real-time frequencies. We evaluate the capabilities of Kinodynamic Fabrics on diverse physically-challenging whole-body control tasks with a bipedal humanoid robot both in simulation and in the real-world. Kinodynamic Fabrics outperforms the state-of-the-art Quadratic Program based whole-body controller on a variety of whole-body control tasks on run-time and reactivity metrics in our experiments. Our open-source implementation of Kinodynamic Fabrics as well as robot demonstration videos can be found at this URL: https://adubredu.github.io/kinofabs.

The Archimedean spiral is a space-filling plane curve that is found in applications ranging from coverage path planning for robot exploration to scan pattern generation for medical imaging. The constant linear velocity (CLV) parameterization of this spiral is of particular interest due to its fixed path velocity and isotropic sampling capability, but the high accelerations near its origin singularity yield poor trajectory tracking that limits its utility. Here, I derive a closed-form time-optimal time scaling for CLV spirals with large path velocities that mitigates the singularity by inspecting the CLV spiral’s acceleration envelope. When applied to a two degree-of-freedom Cartesian scanner, I demonstrate that this approach reduces trajectory tracking error by up to 97.1% as compared to naive CLV spirals with low computational overhead. I further show that this time scaling eliminates the central image distortion near the origin for scanning applications that rely on CLV spirals.

We enable efficient and effective coordination in unpredictable environments, i.e., in environments whose future evolution is unknown a priori and even adversarial. We are motivated by the future of autonomy that involves multiple robots coordinating in dynamic, unstructured, and adversarial environments to complete complex tasks such as target tracking, environmental mapping, and area monitoring. Such tasks are often modeled as submodular maximization coordination problems. We introduce the first submodular coordination algorithm with bounded tracking regret, i.e., with bounded suboptimality with respect to optimal time-varying actions that know the future a priori. The bound gracefully degrades with the environments’ capacity to change adversarially. It also quantifies how often the robots must re-select actions to “learn” to coordinate as if they knew the future a priori. The algorithm requires the robots to select actions sequentially based on the actions selected by the previous robots in the sequence. Particularly, the algorithm generalizes the seminal Sequential Greedy algorithm by Fisher et al. to unpredictable environments, leveraging submodularity and algorithms for the problem of tracking the best expert. We validate our algorithm in simulated scenarios of target tracking.

This paper develops a stochastic programming framework for multi-agent systems where task decomposition, assignment, and scheduling problems are simultaneously optimized. The framework can be applied to heterogeneous mobile robot teams with distributed sub-tasks. Examples include pandemic robotic service coordination, explore and rescue, and delivery systems with heterogeneous vehicles. Due to their inherent flexibility and robustness, multi-agent systems are applied in a growing range of real-world problems that involve heterogeneous tasks and uncertain information. Most previous works assume a unique way to decompose a task into roles that can later be assigned to the agents. This assumption is not valid for a complex task where the roles can vary and multiple decomposition structures exist. Meanwhile, it is unclear how uncertainties in task requirements and agent capabilities can be systematically quantified and optimized under a multi-agent system setting. A representation for complex tasks is proposed: agent capabilities are represented as a vector of random distributions, and task requirements are verified by a generalizable binary function. The conditional variability in capability requirements is used to build a robust model that captures the trade-off between scheduling risk and expected costs. The proposed framework is validated through simulations and real-world testing.

Robotic knee-ankle prostheses have often fallen short relative to passive microprocessor prostheses in time-based clinical outcome tests. User ambulation endurance is an alternative clinical outcome metric that may better highlight the benefits of robotic prostheses. However, previous studies were unable to show endurance benefits due to inaccurate high-level classification, discretized mid-level control, and insufficiently difficult ambulation tasks. In this case study, we present a phase-based mid-level prosthesis controller which yields biomimetic joint kinematics and kinetics that adjust to suit a continuum of tasks. We enrolled an individual with an above-knee amputation and challenged him to perform repeated, rapid laps of a circuit comprising activities of daily living with both his passive prosthesis and a robotic prosthesis. The participant demonstrated improved endurance with the robotic prosthesis and our mid-level controller compared to his passive prosthesis, completing over twice as many total laps before fatigue and muscle discomfort required him to stop. We also show that time-based outcome metrics fail to capture this endurance improvement, suggesting that alternative metrics related to endurance and fatigue may better highlight the benefits of robotic prostheses.

Semantic scene understanding is beneficial for mobile robots. Semantic information obtained through on-board cameras can improve robots’ navigation performance. However, it is challenging to obtain semantic information on small mobile robots with constrained power and computation resources. We propose a new lightweight convolutional neural network that is comparable to previous semantic segmentation algorithms for mobile applications. Our network achieved 73.06% on the Cityscapes validation set and 71.8% on the Cityscapes test set. Our model runs at 116 fps with 1024×2048, 172 fps with 1024×1024, and 175 fps with 720×960 on NVIDIA GTX 1080. We analyze a model size which is defined as the summation of the number of floating operations and the number of parameters. The smaller model size enables tiny mobile robot systems that should operate multiple tasks simultaneously to work efficiently. Our model has the smallest model size compared to the real-time semantic segmentation convolutional neural networks ranked on Cityscapes real-time benchmark and variable lightweight convolutional neural networks. On the Camvid test set, our model achieved an mIOU of 73.29% with Cityscapes pre-trained, which outperformed the accuracy of other lightweight convolutional neural networks. For mobile applicability, we measured frame-per-second on different low-compute devices including a smartphone. Our model operates at 35 fps on Jetson Xavier AGX, 21 fps on Jetson Xavier NX, and 14 fps on a ROS ASUS gaming phone. 1024×2048 resolution is used for the Jetson devices, and 512×512 size is utilized for the measurement on the phone. Our network did not use extra datasets such as ImageNet, Coarse Cityscapes, and Mapillary. Additionally, we did not use TensorRT to achieve fast inference speed. Compared to other real-time and lightweight CNNs, our model achieved significantly more efficiency and optimal trade-off of accuracy, inference speed, and model size.

Trust-aware human-robot interaction (HRI) has received increasing research attention, as trust has been shown to be a crucial factor for effective HRI. Research in trust-aware HRI discovered a dilemma — maximizing task rewards often leads to decreased human trust, while maximizing human trust would compromise task performance. In this work, we address this dilemma by formulating the HRI process as a two-player Markov game and utilizing the reward-shaping technique to improve human trust while limiting performance loss. Specifically, we show that when the shaping reward is potential-based, the performance loss can be bounded by the potential functions evaluated at the final states of the Markov game. We apply the proposed framework to the experience-based trust model, resulting in a linear program that can be efficiently solved and deployed in real-world applications. We evaluate the proposed framework in a simulation scenario where a human-robot team performs a search-and-rescue mission. The results demonstrate that the proposed framework successfully modifies the robot’s optimal policy, enabling it to increase human trust at a minimal task performance cost.

Coordination in multi-agent systems is a vital component in teaming effectiveness. In dynamically changing situations, agent decisions depict emergent coordination strategies from following pre-defined rules to exploiting incentive-driven policies. While multi-agent reinforcement learning shapes team behaviors from experience, interpreting learned coordination strategies offers benefits in understanding complex agent dynamics and further improvement in developing adaptive strategies for evolving and unexpected situations. In this work, we develop an approach to quantitatively measure team coordination by collecting decision time series data, detecting causality between agents, and identifying statistically high coordinated sub-teams in decentralized multi-agent task allocation operations. We focus on multi-agent systems with homogeneous agents and homogeneous tasks as the strategy formation is more ambiguous and challenging than heterogeneous teams with specialized capabilities. Emergent team coordination is then analyzed using rule-based and reinforcement learning-based strategies for task allocation in operations at different demand stages (stress) levels. We also investigate correlation vs. causation and agent over- or under-estimating demand levels.

Though they are compliant, nimble and morphologically intelligent, fluidic soft robots often rely on bulky components for power and actuation. This work contributes a design methodology which enables development of soft fluidic robots that move in a sequenced fashion, enabling lightweight devices with embodied intelligence. Bezier-curved beams were introduced as a design building block whose antagonistic placement results in Representative Auxetic Element (RAE) that can be patterned on inflatable shells. Kinematics and loading behaviour of these design building blocks were studied through Finite Element Analysis (FEA). We give a methodology for patterning RAEs on cylindrical and conic shells to create soft fluidic components that move (motion components) and those that delay fluid flow (pinch components). We verify the physical concepts governing the design methodology through two prototype devices that produce sequenced motion under a single fluidic input. Devices using this framework have the potential to perform complicated sequenced motions with lightweight control components.

We focus on robot navigation in crowded environments. To navigate safely and efficiently within crowds, robots need models for crowd motion prediction. Building such models is hard due to the high dimensionality of multiagent domains and the challenge of collecting or simulating interaction-rich crowd-robot demonstrations. While there has been important progress on models for offline pedestrian motion forecasting, transferring their performance on real robots is nontrivial due to close interaction settings and novelty effects on users. In this paper, we investigate the utility of a recent state-of-the-art motion prediction model (S-GAN) for crowd navigation tasks. We incorporate this model into a model predictive controller (MPC) and deploy it on a self-balancing robot which we subject to a diverse range of crowd behaviors in the lab. We demonstrate that while S-GAN motion prediction accuracy transfers to the real world, its value is not reflected on navigation performance, measured with respect to safety and efficiency; in fact, the MPC performs indistinguishably even when using a simple constant-velocity prediction model, suggesting that substantial model improvements might be needed to yield significant gains for crowd navigation tasks. Footage from our experiments can be found at https://youtu.be/mzFiXg8KsZ0.

We focus on the problem of rearranging a set of objects with a team of car-like robot pushers built using off-theshelf components. Maintaining control of pushed objects while avoiding collisions in a tight space demands highly coordinated motion that is challenging to execute on constrained hardware. Centralized replanning approaches become intractable even for small-sized problems whereas decentralized approaches often get stuck in deadlocks. Our key insight is that by carefully assigning pushing tasks to robots, we could reduce the complexity of the rearrangement task, enabling robust performance via scalable decentralized control. Based on this insight, we built PuSHR, a system that optimally assigns pushing tasks and trajectories to robots offline, and performs trajectory tracking via decentralized control online. Through an ablation study in simulation, we demonstrate that PuSHR dominates baselines ranging from purely centralized to fully decentralized in terms of success rate and time efficiency across challenging tasks with up to 4 robots. Hardware experiments demonstrate the transfer of our system to the real world and highlight its robustness to model inaccuracies. Our code can be found at https://github.com/prl-mushr/pushr, and videos from our experiments at https://youtu.be/nyUn9mHoR8Y.