Sensorimotor collaboration: Dynamics, Control, and Phenomenology

Abstract

This dissertation investigates the interplay between task constraints, partner dynamics, and the phenomenal experience of control in human sensorimotor collaboration. Specifically, it examines how the sense of agency (SoA), control strategies, and motor outcomes evolve in bimanual and interpersonal joint actions—including interactions with artificial agents—across tasks with varying mechanical structures. To enable high-resolution study of these phenomena, I developed CollabSim, a novel, finger-driven simulation platform that allows dyads to apply forces to shared virtual objects governed by Newtonian dynamics via pulley-cable systems. The environment supports task configurations with varying degrees of redundancy and co-dependence, facilitating multi-level analysis of coordination across agentic, inter-limb, and inter-finger dimensions. Study 1 addressed the phenomenological dimension of joint action by modeling subjective control under different task constraints and collaboration contexts (bimanual vs. novice–expert). Participants performed redundant (Guided-Elevator) and co-dependent (Free-Elevator) force-control tasks and reported trial-level sense of control, perceived role, and coordination breakdowns. Bayesian ordinal and categorical models indicated that the phenomenal experience of control was not necessarily diminished in joint actions; in some cases, it appeared to increase in novice–expert pairings, particularly within co-dependent task contexts. The models revealed a graded association between SoA and perceived roles—highest for leadership, followed by dynamically switching roles, we-agency, and lowest for follower roles. These findings suggest that SoA in joint action is shaped by social and contextual factors beyond task success or physical input. Studies 2 and 3 examined behavioral correlates of these phenomenological observations using force signal analysis. Study 2 quantified lateralized control strategies in bimanual solo tasks and found consistent asymmetries, with the non-dominant hand contributing greater force magnitude and variability, while the dominant hand performed more precise, lower-amplitude control. These asymmetries were task-dependent and attenuated in co-dependent conditions. This supports the motor abundance framework, confirming that lateralization is not fixed but dynamically shaped in response to task constraints. Study 3 extended this analysis to novice–expert dyads. Expert involvement yielded smoother and more stable system-level force trajectories, especially in redundant tasks. Importantly, novices exhibited heightened engagement and behavioral stabilization when collaborating with an expert. This supports a model of synergy-based control in which expert behavior reorganizes the dyad’s uncontrolled manifold, enabling distributed stabilization of performance via adaptive redundancy exploitation. Study 4 examined the influence of cognitive framing on joint action with artificial agents. Participants interacted with an identical open-loop agent across conditions framed as “human,” “computer,” or “model.” Despite invariant agent behavior, subjective reports of control and role attribution varied significantly by framing condition. These results suggest that social cognition and expectations modulate phenomenological experience in joint action, particularly in the context of human–AI collaboration. Together, these studies offer a multi-level account of joint sensorimotor behavior, connecting phenomenological, behavioral, and theoretical analyses. The findings advance models of distributed agency, synergy, and socially situated motor control.