Project, Research & Publication

Index

1. Research: 1st System for Automatic 3D Animation of Piano Performances, June 2010 – July 2012

2. Project: Cross-Platform Animation System, June 2011 – December 2012

3. Research: Balance Control in Interactive Motion, March 2012 – December 2012

4. Project: Reconstruction of Criminal Virtual Environment for Dalian Police Station, February 2005 – May 2007

5. Research: Real-time Collision Detection between Rigid Bodies, January 2006 – January 2007

6. Research: Real-time Collision Detection and Response, January 2007 – January 2008

7. Research after M.S. Graduation, January 2008 – November 2009

8. Project: Navigation in Unity-based Application with Intel Mocap Camera - April 2013

9. Project: Kinect Server for MAYA - May 2013

10. Project: A Simple JavaScript GUI Development - May 2013

11. Project: A Simple GLSL GUI Development - May 2013

12. Project: Music Visualization, Implemented with Open ES 2.0 and Java, July 2013

13. Project: A Fighting Motion with Special Effects, Implemented with Cocos2D & Cocos3D Game Engine, Objective C - July 2013

14. Project: Weather Report System, Implemented with Android SDK, SQLite, JSP, MySQL, Struts 2.1.8+Spring 2.5.6+Hibernate 3.6 - September 2013

15. Project: 3D version of Cocos2D-X - September 2013

Note:

* I own all of the source codes mentioned above, but due to copy-right and/or high-tech idea and implementation for future potential benefits, only runnable program is provided in the first 9 projects.

** The tiny projects including Project 10 and 11 was released with all the source codes, which are generally to show my common-used OO programming skills.

1. Research: 1st System for Automatic 3D Animation of Piano Performances, June 2010 – July 2012

  • Role: Sole Developer and Researcher, advised by Professor Bernd Hamann and Michael Neff.
  • System introduction: Given a musical piece, this is the only existing system to automatically generate 3D animations of piano performance.
  • Demos:
  1. Demo
  • Coding skill: C++ for the main codes, MATLAB for fingering generation and optimization computation, Maya API for a basic animation platform, STK for midi import.
  • Related professional software and hardware: Vicon Digital Optical Motion Capture System, Maya.
  • Paper published in a famous computer animation journal as 1st author.Yuanfeng Zhu, Ajay Sundar Ramakrishnan, Bernd Hamann and Michael Neff. Journal CAVW (Computer Animation and Virtual Worlds), 21 SEP 2012. Indexed by SCI and EI.
  • Noted in famous scientific website and magazine: Phys.Org:  Link; New Scientist Magazine: Link
  • A short version of this research is accepted as a finalist in the Written Paper Category in 2013 Interdisciplinary Graduate and Professional Student Symposium, UC Davis. This symposium provides a platform to make this research recognizable by the other professional schools and academic colleges at UC Davis.
  • Download:  Paper
  • Cooperation and many thanks to: My partner Ajay Sundar Ramakrishnan provides continuous help on paper polishing. 3D artist and programmer Paris Maroidis for his quite important contribution to the development of our system. Jonathan Graham for his elegant modeling of the piano character. Yingying Wang and Pengcheng Luo for their help during motion capture process. Professor Hong Zhu for her suggestion related to piano theory. Binglin li’s piano performance for motion capture work. Many valuable suggestions from IDAV group members Yejin Kim, Tyler Martin and Nick Toothman, and Yubo Zhang from VIDI Lab.

2. Project: Mobile Animation System (PhD GSR Funding Source), June 2011 – December 2012

  • Role: Sole Developer, advised by Professor Michael Neff.
  • System introduction: Given a user input such as a sentence in the Android client, the server will automatically parse the sentences, generate a corresponding 3D human motion with various gestures and facial expressions, and then send the animation back to the client. The work mentioned below is now providing kernel re-usable character animation and network components for the research group at UC Berkeley, UC Davis and UC Santa Cruz.
  • Important components implemented:
    1. Motion-capture-data-driven full body animation (using BVH standard to represent motion data)
    2. Script-driven facial animation (using BML standard to represent facial animation)
    3. Real-time stream body and facial animation from server to client.
    4. Real-time communication between Android Client, Unity3D Server (Windows and Android version available), C++ Server (Windows version), C# Server (Windows version). C++ and C# servers provide interfaces to communicate with other C++ and C# based components to enrich system functions, such as to embed an audio recognition module, motion database management system, motion gesture analysis and synthesis module).
  • Demos:
  1. Android Client Demo


  2. Communication Between Android Clients and Unity Server


  3. Communication Between Unity Server, CPP Server and C Sharp Server
  • Coding skills: C#, game engine Unity3D, C++, C++ Interop, Android.
  • Doc: ReadMe
  • Programs: (Please ask me for special version (Windows and or Android versions) of the following programs if needed.)
  1. Facial Animation Control
  2. Android Client
  3. Unity3D Server
  4. C++ Server
  5. C# Server

3. Research: Balance Control in Interactive Motion, March 2012 – December 2012

  • Role: Sole Developer and Researcher, advised by Professor Bernd Hamann and Michael Neff.
  • System introduction: Given a hit to the character in a full-dynamic system, and a desired motion pose with exact timing requirement (optional), the system uses optimization framework to automatically generate a responsive balance motion satisfying various user-requirements on motion styles.
  • Demos:
    1. Demonstration of the advantages of the proposed optimization framework


    2. Demonstration of the advantages of the proposed 1st Lyapunov-Function-based constraint


    3. Demonstration of the advantages of the proposed 2nd Lyapunov-Function-based constraint


      • Important components implemented:
      1. MATLAB to implement a proposed optimization framework to analyze and predict the motion balance state of a candidate motion, and find optimum motion solution.
      2. Use the famous physics engine ODE for human motion simulation, collision detection and response.
      3. Novelty 1: Uses one Lyapunov Function based constraint to quickly (56% better) eliminate candidate solutions which failed to achieve the desired motion goal.
      4. Novelty 2: Uses another Lyapunov Function based constraint to more accurately (36~93% better based on different motion types) and efficiently (3%~24% better) predict motion balance state controlled by a candidate motion.

    4. Project: Reconstruction of Criminal Virtual Environment for Dalian Police Station, February 2005 – May 2007

    • Role: Sole Developer and Researcher, (China NSF GSR Funding Source), advised by Professor Jun Meng.
    • Key Tasks: Upgraded the original system to support physical collision detection and response between environment and characters using ODE physics engine; designed a user-friendly GUI with GLUI GUI library.
    • Coding skills: C++, OpenGL.

    5. Research: Real-time Collision Detection between Rigid Bodies, January 2006 – January 2007

    • Role: Sole Developer and Researcher, advised by Professor Jun Meng.
    • Improved: executive speed with the same accuracy as others’ work by design of a hybrid hierarchical bounding volume (HBVH).
    • Wrote my first paper for the international publication as 1st author "Research on Real-Time Collision Detection Based on Hybrid Hierarchical Bounding Volume."  Journal of System Simulation, ISSN 1004-731X, Volume 20, No.02, Jan. 20, 2008. Indexed by EI.
    • Download: Paper
    • Coding skills: C++, OpenGL.

    6. Research: Real-time Collision Detection and Response, January 2007 – January 2008

    7. Research after M.S. Graduation, January 2008 – November 2009

    8. Project: Navigation in Unity3D-based Application with Intel Mocap Camera, April 2013

    • Role: Sole Developer, selft-directed.
    • Project introduction: This is a small project to experience how to use Intel Mocap Camera to develop a novel interactive way in Unity3D-based application. During preparing for a job, I spend my spare time within 5 days on reading the SDK document, learn the examples, develop my own demo and summarize the evaluation on Intel PC SDK version 1.0.7383.
    • Important components implemented:
      1. Facial Landmarks-based navigation
      2. Palm Landmarks-based navigation
    • Coding skills: C#, game engine Unity3D, Intel PCSDK.
    • Programs: Helicopter Navigation Control (You need Intel Mocap Camera, Intel PCSDK driver to run this program)
    • Please email me for the evaluation document "Problems, New Ideas and Potential Further Development on Unity-version of Intel PCSDK".

    9.Project: Kinect Server for MAYA, May 2013

    • Role: Sole Developer, self-directed.
    • Project introduction: This is a small project to experience how to use Microsoft Kinect Mocap Camera with OpenNI v2.x driver to drive the skeleton animation in MAYA in real time. OpenNI is a famous open source driver for various motion capture devices including Microsoft Kinect. Because OpenNI v2.x is not compatible with OpenNI v1.x, the redesign of the application is needed to support OpenNi v2. This program is designed based on newest version of OpenNI v2.x alpha.
    • Important components implemented:
      1. Algorithm to generate skeleton animation using the motion captured joint world position captured from Kinect.
      2. A motion-capture server framework to potentially connect any motion capture camera to MAYA for real time motion-capture sampling and skelention animation through MAYA API.

    10. Project: A Simple JavaScript UI development, May 2013

    • Role: Sole Developer, self-directed.
    • Project introduction: This is a very tiny project to show my JavaScript OOP style, by developing a simple GUI system.
    • Important components implemented includes: button, check box and text box which is supported with various interesting customizations.
    • Coding skills: JavaScript
    • Git my codes: JavaScript GUI Design
    • Download: Video Demo

    11. Project: A Simple GLSL UI development, June 2013

    • Role: Sole Developer, self-directed.
    • Project introduction: a tiny project to show my C++ OOP style, by constructing a GUI system with GLSL technology, which provides much faster rendering performance with VBO feature of GLSL than the traditional way.
    • Important components implemented includes: GUI framework using GLSL, and button which supports single-texture and double texture-blended shading style.
    • Coding skills: C++, GLSL, OpenGL.
    • Git my codes: GLSL GUI Design
    • Git my Demo: win32 and X64 program Demo

    12.Project: Music Visualization, implenmented with Open ES 2.0 and Java, July 2013

    • Role: Sole Developer, self-directed.
    • Project introduction: a music visualization project to demonstrate my programming experience on Java, OpenGL ES 2.0, and music related-coding skills. This project is developed based on 3D game engine libgdx, which is a Desktop/Android/iOS/HTML5 Java game development framework.
    • Downlaod program: program runnable on Mac OS, and Windows (Java VM required on Windows) Note: This program employs user microphone as input to drive music visualization. Louder music sound makes visualization more sensitive.
    • Coding skills: Java, OpenGL ES 2.0, Eclipse, music programming, libgdx, Android.
    • Download: Video Demo

    13. Project: A Fighting Motion with Special Effects, Implemented with Cocos2D & Cocos3D Game Engine, Objective C - July 2013

    • Role: Sole Developer, self-directed.
    • Project introduction: A demo to show my Objective C programming experience. It implements a skeleton-driven fighter performs martial arts, added with some special effects with particle system. This demo is written in Objective C, based on Cocos2D-iphone and Cocos3D as game engine, OpenGL ES 2 for rendering, supports iOS and Mac OS.
    • Downlaod program: program runnable on Mac OS
    • Coding skills: Objective C, OpenGL ES 2.0, Xcode 4, Cocos2D-iphone 2 and Cocos3D 2, iOS, MAC OS.
    • Download: Video Demo

    14. Project: Weather Report System, Implemented with Android SDK, SQLite, JSP, MySQL, Struts 2.1.8 + Spring 2.5.6 +Hibernate 3.6 - September 2013

    • Role: Sole Developer, self-directed.
    • Project introduction:
    • 1) Android client side: is to automatically get selected weather report from http://rp5.cz/Weather_in_the_world. It employs SAX, PULL, or DOM approaches to parse the XML weather report generated by http://rp5.cz.
    • It can save and manage the obtained weather information on the designed local SQLite database embedded in Android OS.
    • It can also update reports to the designed JSP server with MySQL database, and remotely manage the updated reports on the server using JSON format for transmission on network.
      2) Server side: maganges the weather reports updated from Android client side, uses JSP with Tomcat 7.0, Struts 2.1.8.1 as web framework, Spring 2.5.6 as dependency injection framework, Hibernate 3.6 as database ORM framework, MySQL 5.6.13 as database management system.
    • Coding skills:
    • 1) Android Client Side: Java, Android SDK, SQL, SLQLite, XML, SAX, PULL, DOM, JSON, Eclipse.
    • 2) Server Side: JSP, Struts 2.1.8.1 + Spring 2.5.6 + Hibernate 3.6 framework, JSON, SQL, MySQL, Eclipse.
    • Download: Video Demo

    15. Project: 3D Version of Cocos2D-X - Jan 2014

    • Role: Sole Developer, self-directed.
    • Project introduction:
    • Cocos2D-X is Top 1 famous open source C++ 2D game engine, and the application based on it can be easily ported to different platform such as Android, iOS, MAC OS, Linux.
    • This development makes this 2D game engine to support 3D animation, 3D collision detection & dynamic response by porting Bullet to Cocos2D-X.
      It is implemented based on Cocos2D-X-2.2, which still does not support any 3D animation and physics engine.
    • Coding skills:
    • 1) Android Version: Android SDK 4.3, Android NDK r9, Cygwin 1.7.25-1, Eclipse(ADT version).
    • 2) Windows Version: VC ++ 2012, OpenGL ES 2.0, Cocos2D-X 2.2.x.
    • Download: 3DGameVideo, 3DPhysicsEngineDemo(AndroidDemo WindowsDemo)