Below you can find information regarding data sets that our research group has released or can provide upon request:

Rutgers Underactuated-hand Manipulation dataset (RUM):

Underactuated hands with compliant fingers offer an appealing grasping solution due to their adaptability to objects of uncertain size and shape. Therefore, they enable stable, robust grasps without tactile sensing or prior planning, using open-loop control. Furthermore, open-source underactuated robotic hands, which we rely on in our work, are easy to fabricate and modify. However, due to uncertainties in the manufacturing process, the hands differ in size, weight, friction, and inertia.

Consequently, precise analytical models for such hands are usually unavailable as they are hard to derive. Thus, as described in a previous publication, data-based modeling enables somewhat accurate predictions and can be used for motion planning and closed-loop control. However, such an approach in the context of adaptive hands is rather novel and has not been extensively explored. Therefore, we wish to contribute a set of task benchmarks for standard performance evaluation and further development, along with a benchmarking platform based on simulations and on real data.

The Rutgers Underactuated-hand Manipulation (RUM) dataset is a wide set of real data collected from several underactuated hands (the Model-T42 and Model-O) across various types of objects. The data comprises approximately 300,000 transition points for each object. Furthermore, each object comes with 10 long validation paths that provide substantial coverage of the hand’s workspace and also involve frequent changes of action.

 

Model-T42
Model-T42
Model-O
Model-O

 

 

 

 

 

 

 

Here we provide links to download the dataset, separated by hand model and data category:

Entire Dataset

Only Model-O and T-42 Files

Model-O

Action Sequences: Data 

Model-T42

Transition Data: Discrete, Continuous

Failure Data: Discrete, Continuous

Objects

Mesh Files: Data

Information on how to access the data files can be seen on the project’s GitHub page.

 

YCBInEOAT Dataset

Due to the lack of suitable dataset about RGBD-based 6D pose tracking in robotic manipulation, a novel dataset is developed. It has these key attributes:

  • Real manipulation tasks
  • 3 kinds of end-effectors
  • 5 YCB objects
  • 9 videos for evaluation, 7449 RGBD in total
  • Ground-truth poses annotated for each frame
  • Forward-kinematics recorded
  • Camera extrinsic parameters calibrated

Dataset Page

Object Pose Estimation In Adaptive Hands Dataset

Many manipulation tasks, such as placement or within-hand manipulation, require the object’s pose relative to a robot hand. The task is difficult when the hand significantly occludes the object. It is especially hard for adaptive hands, for which it is not easy to detect the finger’s configuration.

Dataset Page

 

Rutgers APC RGB-D Dataset

To better equip the research community in evaluating and improving robotic perception solutions for warehouse picking challenges, the PRACSYS lab at Rutgers University provides a new rich RGB-D data set for warehouse picking and software for utilizing it. The dataset contains 10,368 depth and RGB registered images, complete with hand-annotated 6DOF poses for 24 of the Amazon Picking Challenge (APC) objects (mead_index_cards excluded). Also provided are 3D mesh models of the 25 APC objects, which may be used for training of recognition algorithms.

Rutgers Extended RGBD dataset

Dataset download link: download

For each scene in the dataset, we share:

  • RGB Image
  • Depth Image
  • Segmentation mask
  • Parameters
    • camera_pose: pose of the camera in a global frame.
    • camera_intrinsics: intrinsic parameters of the camera.
    • rest_surface: pose of the resting surface such as a table or shelf bin.
    • dependency_order: physical and visual dependency of objects upon each other.
    • pose: ground-truth object pose in a global frame.

Autonomous data generation to train CNNs for object segmentation

The training dataset is generated by physical simulation of the setup in which the robot operates. The tool we developed for autonomous data generation, labeling and training is shared below.

Dataset Generation toolbox: https://github.com/cmitash/physim-dataset-generator

Learnt models for object segmentation
Faster-RCNN (VGG16) Physics Simulation + Self Learning (Shelf): download
Faster-RCNN (VGG16) Physics Simulation + Self Learning (Table-top): download

6D pose hypotheses dataset for manipulation planning in table setup

The 6D pose hypotheses dataset for manipulation planning is generated by

(1) Image taking via an Azure Kinetic camera

(2) A fully convolutional neural network which returns object classification and segmentation probability maps

(2) A geometric model matching process which returns pose hypotheses for each detected object

Examples of scenes (The 1st row shows the RGB images and the 2nd row shows the visualization of pose hypotheses in physics simulator PyBullet)

The dataset can be referenced here.