November 15, 2013 § Leave a comment
My latest reading in Segouat has been the sections on variation in timing of components that don’t vary in value and his proposed model of coarticulation in LSF in the domain of train announcements. Segouat compares variations in duration by the announcement, component (gaze, dominant hand, non-dominant hand), and segment. There’s a lot of enumerating of the data that basically comes down to it’s noisy, some are shorter some are longer, often with variations of more than 10 or 20%. He does however propose a few generalizations. At the announcement level he observes that the informational announcements showed less variation in duration of components than incident announcements. From his comparison of components he notes that gaze durations are likely to be longer in context while the duration of handshapes is reduced. From the segments he remarks on how the stations and numbers are faster in context, possibly because the precision makes recognition easier.
He then makes a brief proposal for a model of coarticulation in LSF train announcements:
- Analyze the semantics of the announcement for key information, these signs will be precise, and if relaxed in isolation will be more tense and precise in context.
- Stations and numbers are signed precisely taking into account any variations in signs for certain stations.
- Take into account phonetic context, the previously determined handshake effect the ones around them, (cause a slight increase in tension).
- Add in any influence the dominant hand has on the non-dominant hand.
In discussing the model he also makes a list of the different aspects considered in the model.
- Semantic context
- Eye gaze raised
- Add gazes to points in space (in anticipation of the hands signs)
- Suppression of signs
- Coarticulation (phonetic context)
- Dominant hand influence on non-dominant hand
- Gaze direction
- Minimum delay between two identical gaze directions
November 1, 2013 § Leave a comment
I’ve finished the section of Segouat’s dissertation analyzing how the variations in value of the handshape, correspond to the parts of speech of the sign those handshapes occur in. Segouat finds that the modifications in handshape due to coarticulation were found only in the lexical signs, and that handshapes that were modified as parts of lexical signs were not modified when used as numbers or as an iconic sign. He hypothesizes that this is due numeric signs and fingerspelling requiring precise handshapes in order to be clearly understood and thus have minimal dependence on context.
October 25, 2013 § Leave a comment
I’ve continued making progress with reading Segouat‘s dissertation and finished his discussion on how the variations in gaze and handshape breakdown by segment. He divides the different segments of the announcements into categories based on whether they vary between their isolated and in context forms and the consistency of variations (don’t vary, vary but always the same variations, don’t vary or one variation, mostly consistent variations, and variations dependent on context). The segments with the least variation tend to be the numbers, stations, and platforms. The segment Segouat classifies as varying depending on context is “votre attention s’il vous plait” (your attention please). Its variations in handshape are strongly dependent on which context the segment is placed and thus coarticulatory.
October 19, 2013 § Leave a comment
SLTAT was this weekend and the conference went really well. The attendees were from a variety of projects around the world focusing on sign language translation and animation. It was interesting to hear about the different projects and approaches to tackling the problems. At DePaul we’ve been focused on improving avatar quality and creating algorithms to create natural motion based on sparse key hand animations of individual signs. Other groups are working on using motion capture data for sign animation, both with a sophisticated multiple camera set up and using the Kinect. The French group, LIMSI (who Segouat worked with) is currently working on automating the transformation of sentences in French to a syntax representing French Sign Language that can then be feed to an animation engine. This is an area we haven’t really addressed, currently to create animations our system requires the animator to determine the correct signs and syntax as well as manually add affects, blinks, etc. We are hoping to work with LIMSI on a project that combines their experience with the translation aspect with our work on producing natural animation. The poster presentation went well and the attendees seemed interested in the work we’ve done.
October 16, 2013 § Leave a comment
Melissa and I have decided to make a video to accompany our poster like Farah and I did for SOCRS last year. We are including one of the timing comparisons Farah made as well as turntables comparing the initial and optimized versions of the eyelashes, teeth, and lighting setup and an animation of some of the new expressions Melissa has been creating with the revised model. Thus I’ve been working on setting up the turntables in Max and rendering them, as well as compositing the video in Aftereffects, and figuring out rendering settings that will work on my tablet’s video player.
October 11, 2013 § Leave a comment
The SLTAT, Sign Language Transcription and Avatar Technology conference that DePaul is hosting is fast approaching and I’ve been working on preparing the poster Melissa and I will be presenting at the event. Our paper, Towards a Real Time Signing Avatar Using Linguistic Principles for Nonmanual Signals, places the work Farah and I did last year on optimizing the teeth and eyelashes in the linguistic context (explaining why one cares about the eyelashes or teeth in a signing avatar) and also includes the work Melissa did this summer on optimizing our lighting setup. I think I’ve figured out how to summarize an eight page paper in a single poster, though I thought we might run out of space. Now we just need to find or render pictures to show the changes we’ve made.
October 4, 2013 § Leave a comment
The latest in Segouat has been on the effects of coarticulation in the dominant and non-dominant hand. Like with the section on eye gaze in Segouat’s analysis many of the variations are due to semantic effects, for instance, use of different signs or a rephrasing of the statement that eliminates signs. He does decide that some of the modifications, insertions and deletions he observes are due to coarticulation. He categorizes the modifications in handshape as more tense or more relaxed, and from his analysis decides that the increased tension in some handshapes is due to the semantic context, they are part of numbers, train stations or signs conveying important information, while the less critical signs have a more relaxed handshape in context. While these changes are not judged to be coarticulatory, Segouat observes that there is a preservation effect, if the preceding sign is more tensed or relaxed the succeeding will have a slight tension or relaxation. Segouat also observes an assimilation between signs with repeated handshapes and the following sign. Many of the variations in non-dominant handshape are also judged be not coarticulatory. Segouat does observe that the tension or relaxation of the dominant hand is echoed in the non-dominant hand.