FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation
CVPR 2025 Highlight
Kefan Chen1,2* Chaerin Min1* Linguang Zhang2 Shreyas Hampali2 Cem Keskin2 Srinath Sridhar1
1Brown University 2Meta Reality Labs
Paper Project Page Code (Coming in June)
1. Upload a hand image to repose or choose one below 📥
Step 1. Tell us if this is right, left, or both hands.
Step 2. Click on image to provide hand keypoints for right hand. See "OpenPose Keypoint Convention" for guidance.
Step 2. Click on image to provide hand keypoints for left hand. See "OpenPose keypoint convention" for guidance.
Step 3. Hit "Done" button to confirm.
OpenPose Keypoints Convention
2. Upload a hand image for target hand pose or choose one below 📥
Step 1. Tell us if this is right, left, or both hands.
Step 2. Click on image to provide hand keypoints for right hand. See "OpenPose Keypoint Convention" for guidance.
Step 2. Click on image to provide hand keypoints for left hand. See "OpenPose keypoint convention" for guidance.
Step 3. Hit "Done" button to confirm.
OpenPose Keypoints Convention
3. Press "Run" 🎯
⚠️ Currently, Number of generation > 1 could lead to out-of-memory
1. Upload a malformed hand image or choose one below 📥
To crop, click top left and bottom right
2. Brush wrong finger
(⚠️and surrounding area)
3. Target hand pose
① Tell us if this is right, left, or both hands
② Click 21 keypoints on the image to provide the target hand pose of right hand. See the "OpenPose keypoints convention" for guidance.
② Click 21 keypoints on the image to provide the target hand pose of left hand. See the "OpenPose keypoints convention" for guidance.
OpenPose keypoints convention
4. Press "Run" 🎯
| Input Image | Image Brushing | Target Hand Pose | Classifier free guidance scale | Seed |
|---|
⚠️ If brushed image doesn't load, please click the example again
Part of this work was done during Kefan (Arthur) Chen’s internship at Meta Reality Lab. This work was additionally supported by NSF CAREER grant #2143576, NASA grant #80NSSC23M0075, and an Amazon Cloud Credits Award.
Citation
If this was useful, please cite us! ❤️
@article{chen2024foundhand,
title={FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation},
author={Chen, Kefan and Min, Chaerin and Zhang, Linguang and Hampali, Shreyas and Keskin, Cem and Sridhar, Srinath},
journal={arXiv preprint arXiv:2412.02690},
year={2024}
}
If error persists, please try the following steps:
1. Refresh the page and try again.
2. The issue might be due to compatibility with HuggingFace or GPU memory limitations. We recommend cloning this repository and trying it with your own GPU if possible.
3. Kindly leave a message on our HuggingFace Spaces Community tab (located at the top right), on our GitHub repository's Issues page, or send us an email. We are happy to help you as soon as possible.
If the result is not satisfactory:
1. Try changing either Classifier Free Guidance Scale or Seed, that can be found at "More Options".