CREU 2013 – Fall 2013 – Week Two

September 27, 2013 § Leave a comment

I’ve continued reading Segouat’s dissertation about coarticulation in French Sign Language.  After recording video data of a person signing train announcements and sections of those announcements, and comparing the in context and isolated utterances of variations he analyzes the results by their component (gaze direction, dominant hand shape, non-dominant handshape), part of speech, and timing.  In the section on gaze direction he identifies multiple variations between the in context and isolated gaze directions, however in his analysis he decides that many of the variations are not due to the phonetic environment but due to the semantic context (for instance adding emphasis to parts of the utterance) and thus not coarticulatory.  The one variation that he observes and decides is due to coarticulation is an elimination of a second gaze to a certain point shortly after the initial.  Segouat hypothesizes that this is due to a minimum time between two gazes to the same point.

Advertisements

CREU 2013 – Fall – Week One

September 12, 2013 § Leave a comment

School’s back in session and I’m back at work with DePaul’s ASL synthesizer project.  On the agenda for this year is improving our ASL animations through the creation of a model of coarticulation in ASL that will help generate more natural and realistic movement.  Over the summer I conducted a review of studies examining coarticulation in signed languages.  Currently I am finishing “Modelisation de la coarticulation en Langue des Signe Francaise pour la diffusion automatique d’informations en gare ferroviaire a l’aide d’un signeur virtuel” by Jeremie Segouat which examines the variations in signing between isolated signs and signs in context.

In addition to the information we have gathered from previous studies on coarticulation in signed languages we will be analyzing the corpus published by the ASL Linguistic Research Project at Boston University.  Using this data we will be examining the transitions between signs and how the transitions change as the distance between sign positions increases (do the hands move faster so the duration of the transition remains constant? are the transitions longer because the hands move at the same velocity? some of both? something different?).

In October, DePaul University will be hosting SLTAT 2013 (Symposium on Sign Language Translation and Avatar Technology).  At the conference Melissa and I will be presenting a poster, “Toward a Real-Time ASL Avatar Utilizing Linguistic Principles for Nonmanual Signals,” based off the work Farah and I did last year and that Larwan, Melissa and I did this summer.

Spring Quarter – Weeks One – Three

April 24, 2013 § Leave a comment

Over the past few weeks I have been working on creating the final version of the teeth texture and replacing the preliminary texture.  The preliminary texture had places where the profile of the teeth was uneven and the value shifted sharply from the compositing.  In addition the transparency maps were fuzzy.

TeethTexture_InitalProblems

The problems in the original textures are a result of the low number of sub images and the large overlap between subsequent images.  For the final versions I increased the number of renders taken around the teeth from ten to sixteen and decreased the size of the rendered image so there was a minimal overlap when the images were joined.   I first attempted to calculate the precise field of view needed to for each section.  However, due to the irregular shape of the teeth there were gaps in the result.

TeethTexture_v2Problems

Instead of trying to guess the proper field of view I constructed a grid and rendered the images with a large enough field of view to show the entire grid section.  For each segment I rendered two images, one with the grid and one with just the teeth.

TeethTexture_v3Setup

I used the version with the grid to crop the just teeth renders then joined the edited versions to form the panorama.  The new version had much less noticeable seams than the original however there was still a slight discontinuity in a couple places.  I fixed these areas with a little work in Photoshop.  The final result is a definite improvement.

TeethComparison

The new versions of the texture don’t quite line up with the old ones so the UVs need a little adjustment.  I am currently working on this.

In addition to the work on the teeth Farah and I have prepared and submitted an abstract for DePaul’s School of Computing Research Symposium.  We are hoping to present a poster on our work.

Winter Quarter – Weeks Six – Nine

March 20, 2013 § Leave a comment

It’s been a busy few weeks since I lasted posted.  Week six, (2/15) we didn’t meet because Dr. Wolfe was out of town.  Over weeks six and seven of the quarter I experimented with closing off the gaps at the top of the teeth.  The problem is that when using a transparency map you get the profile you want on the sides of the object, but unless the object is 2D any holes will show the hollow interior.  As previous pictures have shown this isn’t a huge problem.  Mostly the teeth will be viewed from the front, and at some distance.  However, if Paula opens up wide we need to be able to see at least a reasonably whole top of the molars.

ToungeTeethOptimize_9pic2

To address this problem I first experimented with adjusting the transparency map.  By moving the image up and down slightly I was able to change the locations of the transparent sections on the model and decrease the gap between the front and back of the teeth.  Making the material of the teeth two sided also helped because instead of seeing the background y0u could see the back of the teeth, decreasing the apparent size of the hole.

ToungeTeethOptimize_9pic6_alphaMoreDown

I also experimented some with creating a second plane over the gaps in the teeth with its own transparency map, however, the pieces didn’t fit together well.  In discussion with Dr. Wolfe and Dr. McDonald we decided that adjusting the alpha map would be sufficient to conceal any holes in the teeth, especially with two sided materials and the interior of the mouth in place.

Having more or less finished with the top teeth, they still need a few final adjustments but I am going to do that after I have the bottom teeth up to about the same point.

I have constructed the geometry and rough texture maps for the bottom teeth and started working on the UVs.  I have to finish editing the UVs on the bottom teeth, then I will go back and make the final texture maps for both sets of teeth.  After seeing how the new teeth look with Paula I will make any final adjustments.

ToungeTeethOptimize_9pic9

Winter Quarter – Weeks Four and Five

February 15, 2013 § Leave a comment

We were unable to meet two weeks ago due inclement weather, so over the past two weeks I have been working on refining the texturing of the new teeth model and seeing how well the new teeth integrate with Paula.  With the new and improved texturing the teeth are looking really good.  Can you tell which are the new version and which are the original?  Answer:  The top version are the new teeth.

Answer:  The top version are the new teeth. My work on the texture coordinates has eliminated most of the problems with the stretched gums and teeth running off the edge of the model seen in my previous post.

I also started experimenting with a transparency map.  This will stop max from rendering the portions of the new model that are between or outside the teeth.  I still need to create a final version of the transparency map, however this quick version from the texture shows that the transparency mapping should work well.  ToungeTeethOptimize_7pic7

The top is the new teeth before the transparency mapping, the middle is after the transparency mapping, and the bottom is the original teeth for comparison.  As you can see the transparency map eliminates the black areas around the teeth, which allows us to have the slight indentations without making the edge of the teeth entirely defined by geometry (thus using a lot more polygons).  Since this is a rough map the edges are a bit feathered, however a better map will easily solve this problem.

One of the major issues remaining was the difference in brightness.  Looking at the teeth by themselves there is a definite but not too obvious difference.  The new teeth are slightly darker.  This became an obvious problem when I inserted the new teeth into the model of Paula.  Expressions_Comparison

By themselves the new teeth seem dark but not too bad, however when seen with the old version of the bottom teeth its a little scary, although as the final image shows the settings on the old teeth are a little on the bright side.

After consulting with Dr. Wolfe we decided to add a little self illumination to the teeth to brighten them up.  This produced some good results.  Expressions_SelfIllumComparisonAs the image shows adding a self illumination map with the texture image eliminates the darkness of the new teeth.  With the self illumination to 100% the teeth start to get a little to bright, toning it down a little helped.  Using self illumination is good because we can easily control the brightness as a function of distance.  We can increase the self illumination as the distance increases to help the teeth stand out. Expressions_pic8

Here’s another picture of the new teeth.

For next week I will be working on creating a final alpha map for the teeth and start working on making the bottom teeth.

Farah’s been making good progress with the lashes.  LashesOnPaula5Here’s what they look like now.  Once we finish the new teeth and eyelashes we will have substantially reduced the number of polygons in Paula.  I’m not sure of the exact numbers but  we will have easily eliminated over 20,000 polygons (in a 67,000 polygon model).

Winter Quarter – Week Three

January 28, 2013 § Leave a comment

I have continued working on constructing a new model of the teeth.  I took the individual images of the teeth and used Photoshop to construct a panorama.  The results are promising.  TeethTexture1There is some perspective distortion between the different images so the different images don’t blend together totally smoothly, I think that taking more images, each with a smaller field of view and using Photoshop’s masking tools to help blend from one image to the next will solve these issues.

After making the panorama texture I tried applying it to the teeth.

TeethTexture1_ComparisonIt’s a good start, they definitely look like teeth, however there is some distortion of the texture from the default uv coordinates.  Over the next week I need to spend some time editing the texturing, theoretically a cylindrical map will work well.  I’ve modified the teeth so that they have a back side.  This presents another problem with the default uv coordinates, currently the back side of the teeth is black.
ToungeTeethOptimize_6pic3

After I get the texture coordinate issues sorted out I will see how the new teeth look when they are actually in Paula’s mouth and identify further issues that need to be addressed.

Winter Quarter – Weeks One and Two

January 25, 2013 § Leave a comment

We’re back from winter break and continuing to work on reducing the number of polygons in Paula.  I have begun working on remodeling the teeth.

I started by constructing a new rough model of the top set of teeth.  ToungeTeethOptimize_5pic1It is basically a cylinder that’s been edited slightly to fit the curvature of the existing teeth more closely.  I am going to add a back to the new model so the teeth are fully 3D and don’t start to disappear if Paula opens really wide.

However even tripling the face count of the new teeth will still result in a large decrease in the number of polygons.  ToungeTeethOptimize_5comparisonAs you can see the new teeth have a lot fewer polygons.  The original top teeth have 1930 polygons, my replacement model has 55 polygons currently.

Instead of geometry to add the detail to the teeth I am planning on using texturing, bump mapping and transparency mapping to add realism.  I am working on a script that moves a camera around the teeth and renders the image.  I will then combine the different images to create a single panorama texture image will be applied to the new model with a cylindrical projection.  TeethTexture1Here is a set of the rendered images laid out as a panorama.  This is rendered at 10 points around the teeth to keep the number of images manageable while I work on the script.  For the final version I would like to take the renders in smaller pieces so there is a less noticeable change of perspective.

Among the options for rendering in 3DS Max is the ability to render the z -depth.  In a z-buffer rendering system the computer tracks the distance from the camera to the closest object at each pixel.  If an object is closer than the distance than it is visible at that pixel, if it is farther it is occluded.  3DS Max allows you to render out the z-depth information as a separate image file.  Z DepthTeethThis is a sample z-depth render of the front of the teeth.  I am currently working on modifying my script to render z-depth images in addition to the normal color render at each point on the camera’s path.  Adding bump mapping will make the surface appear to have separate, slightly curved teeth.  By changing the near and far values on the z-depth render you can obtain varying levels of detail.  By increasing the far value it is possible to obtain a black and white image of the teeth to use for transparency mapping.  The new teeth model will only be shown where the image is white, creating the proper profile for the teeth.