tactile-slip-detection-pze

Tactile Slip Detection from High-Bandwidth Tactile Sensing

Learning-based detection of incipient slip from piezoelectric tactile vibrations, with robustness to manipulation perturbations and real-time performance.

🚀 Runnable demo
👉 https://github.com/thayral/tactile-slip-demo
Slip detection from piezoelectric tactile spectrograms (FFT + GRU), with example data and inference pipeline.

Main contributions

C1 — Early slip detection from tactile vibrations
Detect incipient slip using a piezoelectric tactile sensor capturing friction-induced vibrations.
Slip cues are extracted through learning-based spectro-temporal analysis and classified in real time (100 Hz) with short reaction delay.

C2 — Data-driven robustness to perturbations
Improve robustness to transient events and actuation noise through perturbation-aware training.
This significantly reduces false alarms (robustness: 38.77 % → 90.43 %) while preserving perfect recall on slip events and low detection latency (24.1 ms average).

Context project

TraceBot use-case & platform context
Learn more on my PhD page
(setup, sensors, demos)

Slip detection must be early, reliable, and robust to perturbations. While slip generates characteristic high-frequency tactile dynamics, real manipulation introduces many slip-like events (actuation noise, force transients) that can cause false alarms.

Core method

Spectro-temporal features (PzE → FFT/PSD → Spectrogram)

We process high-bandwidth PzE tactile signals in short windows, extract frequency-domain PSD features via FFT, and build a spectrogram for slip classification.

Identify spectral patterns of friction
Analyse temporal evolution with recurrence
100Hz classification with binary classes
Training with binary cross-entropy (BCE)

Dataset 1 — Controlled slip characterization

To study the intrinsic tactile signature of slip under controlled conditions, we first collect data on a dedicated characterization bench.
The goal is to validate the representation, architecture, and detection latency, independently of manipulation complexity.

Characterization bench

Piezoelectric tactile sensor mounted on a flat surface
A robotic probe applies a controlled normal force
Slip is generated through programmed sliding motions
Slip onset timing is obtained from ground-truth probe position

Dataset generation

Slip trajectories are randomly parametrized to cover a wide range of contact conditions:

Normal force: 2–10 N
Sliding distance: 10–32 mm
Sliding speed: 200–2000 mm/min
Typical slip duration: ≈ 1.5 s

Dataset size: 3,200 recordings

Detection performance (controlled conditions)

Slip detection visualization on characterization bench

On this controlled dataset, the FFT–GRU model achieves:

Accuracy: 98.73%
F1-score: 0.9787
Average detection delay: 8.5 ms

These results demonstrate that high-bandwidth tactile vibrations contain sufficient information for early slip detection, and that the proposed spectro-temporal model can exploit them in real time.

This controlled benchmark establishes a reference point for detection performance before addressing robustness under manipulation-induced perturbations.

From controlled slip to embodied perturbations

While the characterization bench provides clean ground truth and controlled slip trajectories, real robotic manipulation introduces additional sources of tactile dynamics that are not related to slip.

During grasping and manipulation, tactile sensors are exposed to:

actuation-induced vibrations,
transient force variations,
contact reconfigurations and load changes,

all of which can produce vibration patterns that resemble slip at the signal level.

As a result, models trained only on controlled slip data may exhibit false alarms when deployed on a robot, despite performing well on idealized benchmarks.

To address this gap, we move from controlled tactile characterization to embodied data collection under manipulation, and explicitly model non-slip perturbations during training.

Manipulation perturbations and false slip cues

During manipulation, tactile signals are affected by multiple sources of perturbations. Some are transient, others persistent, and they can originate either from the environment or from the robot itself. These events often produce vibration patterns that resemble slip, leading to false alarms if not properly modeled.

Transient perturbations Short, impulsive events Broadband vibration spikes		External disturbance (environmental contact)	Intentional action (regrasp, squeezing)
Ambient perturbations Persistent vibrations Narrow-band spectral content		Environmental vibration (tool, surface)	Actuation noise (motor vibration)

These perturbations motivate explicit modeling and supervision strategies to distinguish true slip from slip-like tactile events during manipulation.

To evaluate slip detection under real manipulation conditions, we collect a second dataset directly on a robotic gripper.
This dataset captures both true slip events and non-slip perturbations that arise during grasping and interaction.

The objective is to assess and improve the robustness of slip detection when tactile sensing is embedded in a closed-loop robotic system.

Robotic data collection bench

Multi-fingered robotic gripper instrumented with hybrid tactile sensing
Objects grasped under varying conditions
Slip events generated by the external bench
Additional tactile dynamics arise from actuation, force modulation, and contact changes

Ground truth labels distinguish slip from non-slip perturbations, enabling targeted supervision.

Perturbation modeling

Perturbation taxonomy under manipulation

During manipulation, several classes of perturbations are explicitly introduced:

Δq — Actuation noise: vibrations induced by finger motion and tendon actuation
ΔF_n — Grasp effort variations: transient changes in normal force
ΔF_t — External load variations: tangential forces applied to the grasped object

These events can generate tactile responses that resemble slip, and must be modeled during training to reduce false alarms.

Learning from perturbations

To reduce false slip detections under manipulation, we introduce perturbation-aware supervision during training.

Rather than treating all non-slip samples equally, we distinguish between:

clean no-slip segments, and
non-slip perturbations that generate slip-like tactile dynamics.

This enables the model to learn what should be ignored during manipulation, without sacrificing early slip sensitivity.

Supervision strategies

Perturbations are rare and short events, underrepresented in the data. We adopt targeted learning strategies.

Perturbations imbalance

Perturbation-weighted loss (ω)

\[ \mathcal L_w = - \frac{1}{\sum_t \omega_t} \sum_t \omega_t \log p_t^\star \]

When perturbation time labels are available, training samples are reweighted to balance slip, clean no-slip, and perturbation events.
This explicitly penalizes false alarms caused by actuation noise and force transients.

Focal loss (γ)

\[ \mathcal L_{\mathrm{focal}} = -\frac{1}{N}\sum_t (1-p_t^\star)^{\gamma}\log p_t^\star \]

As an alternative that does not require perturbation labels, focal loss emphasizes difficult predictions by down-weighting easy examples.
This provides a label-free robustness mechanism, a lower-cost alternative.

Haptic data fusion (tactile + proprioception

Haptic signal

The model’s spectrogram input is complemented with the proprioceptive signal of joint torques, estimated from the motor currents (through backdrivability). This enables better disambiguation of intentional events and true slip signal.

These strategies allow the same FFT–GRU architecture to transition from controlled slip detection to robust embodied perception.

Robustness via targeted supervision

Using this dataset, we explore supervision strategies that explicitly incorporate perturbation information during training, while preserving real-time operation.

Model	Delay (ms)	Clean F1	Δq	ΔF_n	ΔF_t
FFT–GRU baseline	17.8 ± 9.5	1.000	52.8	43.9	19.6
FFT–GRU focal (γ = 2)	25.3 ± 21.2	0.998	65.0	50.7	36.8
FFT–GRU weighted (ω)	22.5 ± 16.2	1.000	96.8	56.7	97.0
FFT–GRU haptic (ω + τ)	24.1 ± 18.0	1.000	96.8	79.4	95.1

Key observations

Targeted weighting significantly improves robustness under Δq and ΔF_t (≈ 97% specificity)
Focal loss offers a label-free alternative, but with reduced robustness
Haptic-aware supervision improves resistance to grasp-force variations (ΔF_n)

This embodied dataset bridges the gap between controlled tactile characterization and real robotic deployment, enabling slip detection that remains reliable under manipulation-induced perturbations.

Visualization of false alarm and robustness to perturbation ΔFₙ

Robustness under grasp force variations (ΔFₙ)

Robustness results (FFT–GRU)

Model	Delay (ms)	Clean F1	Δq	ΔF_n	ΔF_t
FFT–GRU baseline	17.8 ± 9.5	1.000	52.8	43.9	19.6
FFT–GRU focal (γ = 2)	25.3 ± 21.2	0.998	65.0	50.7	36.8
FFT–GRU weighted (ω)	22.5 ± 16.2	1.000	96.8	56.7	97.0
FFT–GRU haptic (ω + τ)	24.1 ± 18.0	1.000	96.8	79.4	95.1

Weighting improves robustness under Δq / ΔF_t (specificity ≈ 97%).
Focal loss is a lower-performing alternative without perturbation labels.
Haptic data improves robustness under ΔF_n (up to 79.4%).

Publications

AIM 2023 (published)
Spectro-Temporal Recurrent Neural Network for Robotic Slip Detection with Piezoelectric Tactile Sensor
Théo Ayral, Saifeddine Aloui, Mathieu Grossard
IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2023
Seattle, USA

(In preparation)
Robust Tactile Slip Detection under Manipulation Perturbations
Théo Ayral, Saifeddine Aloui, Mathieu Grossard

Contact

Théo AYRAL

➡️ This work is part of the PhD thesis
Learning-based slip detection for adaptive grasp control
CEA (Leti & List) · Université Paris-Saclay
https://thayral.github.io/PhD-manipulation/