- Speech Rate Awareness System for Non-native Speakers
With the internationalization of cooperative activities, opportunities for conversations between native speakers (NSs) and non-native speakers (NNSs) are increasing, but conversations between NSs and NNSs are not always easy. This paper proposes a Speech Speed Awareness System to support conversations between NSs and NNSs by displaying a screen when the speech rate of the NS becomes too fast for the NNS. We analyzed the speech behaviors of NSs and NNSs from a conversation experiment of word-association games conducted to examine the effectiveness of the proposed system. Although no significant differences in the frequency of NNSs asking back to NSs in…
- Bi-directional Telepresence Robots
A video communication system “Being Here System (BHS)” provides a feeling of being in the same room with a remote user when conversing. Different from a conventional videoconferencing system that displays the raw image from the remote camera, BHS displays the image that is processed in realtime to match with the local site. It extracts the remote user’s figure from the raw image. It also takes the image of the local site that is hidden by the display. Then it superimposes the user’s figure over the image of the local site, and displays this composite image. We confirmed this method…
- Media Enhanced Communication for Second Languages
With a little help from my native friends: Keyword typing improves mutual understanding in L2 conversation. References Hiromi Hanawa, Xiaoyu Song, Mengyuan Tang, Tomoo Inoue, Speakers’ empowerment with keywords: the speaking competency development for multilingual to contribute in a real-time interaction, Collaboration and Technology, LNCS 10391, pp.148-163, doi>10.1007/978-3-319-63874-4_12, Springer, 2017.8. Hiromi Hanawa, Xiaoyu Song, Tomoo Inoue, Keyword generation by native speaker is quick and useful in conversation between native and non-native speaker, Proceedings of the 2017 IEEE 21th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2017), pp.145-150, 2017.4. Hiromi Hanawa, Xiaoyu Song, Tomoo Inoue, Key-Typing on teleconference:…
- Automatic Speech Recognition for Non-native Speakers
We proposed a method to increase the recognition rate of non-native speakers’ speech by inferring non-native speakers’ speech from the native speakers’ speech, which has higher recognition rates.
- Wearable Cuing System for Physical Collaboration
This research goal is to facilitate real-time serial cooperative work. In real-time serial cooperative work, the order of subtasks is important because a failure of the order leads to a failure of the whole task. So, coordinating the order of workers’ subtasks is necessary to smoothly accomplish the tasks. In this paper, we propose a method that uses vibration to present the action order and coordinate the orders. We conducted a user study to verify the effectiveness of the method in theatrical performance practice, which was an example of real-time serial cooperative work. As a result, order coordination reduced mistakes…
- Super-real Experience by Physical+VR Objects
Super-real experience can be realized by combining the objects in both physical and VR environments. We are working on using such kind of super-real experience system as a safety training system. To keep safe, it is necessary to know what the dangers are. But it’s hard to understand them just by listening to lectures. On the other hand, it is also risky to experience the dangers in the real world. Here we use a system that combines objects in both physical and VR environments to provide a safe and realistic way to experience dangers.
- VR Psychotherapy System
Immersive self mental healthcare system has been developed based on a visual counseling method. References Asuki Nakanishi, Yasuhito Noguchi, Toshiya Nakajima, Zhen Yang, Atsuko Matsumoto, Takeshi Kamita, Tsunetsugu Munakata, Tomoo Inoue, A VR self mental healthcare system by SAT-based method, IPSJ SIG Technical Report, Vol.2017-DCC-15, No.35, pp.1-8, 2017.1. Asuki Nakanishi, Yasuhito Noguchi, Toshiya Nakajima, Zhen Yang, Atsuko Matsumoto, Takeshi Kamita, Tsunetsugu Munakata, Tomoo Inoue, Development of a SAT-based self mental healthcare system adopting immersive head-mounted display, VRSJ Research Report, Vol.21, No.CS-4, pp.7-10, 2016.12.
- Being Here System: High-presence Video Communication
A video communication system “Being Here System (BHS)” provides a feeling of being in the same room with a remote user when conversing. Different from a conventional videoconferencing system that displays the raw image from the remote camera, BHS displays the image that is processed in realtime to match with the local site. It extracts the remote user’s figure from the raw image. It also takes the image of the local site that is hidden by the display. Then it superimposes the user’s figure over the image of the local site, and displays this composite image. We confirmed this method…
- Multiple Auditory Localization for Multi-Participant Remote Conversation
In multi-participant conversations, we can hear the voices from multiple directions, which can be helpful with listening comprehension. In the case of remote conversations, we use simple 2-channel stereo acoustic technology to let the user be able to hear voices from multiple directions. References 野口康人, 叶璟, 成合智子, 井上智雄, 多人数遠隔会話のための音像定位分散の効果, 情報処理学会論文誌：デジタルコンテンツ, Vol4, No.1, pp.26-36, 2016.2.
- A Drawing Generation by Crowdsourced Microtasks
An illustration is a key factor to attract document readers. However, not everybody can make it in good quality. Thus illustration has a high demand. Although conventionally clients request skilled creators to draw illustrations with good rewards, there exist huge needs for less expensive illustrations. We propose a novel method of generating such illustrations by crowd-sourced microtasks. In this method, a basic drawing is generated by unskilled crowd workers from one photograph. Experimentation to study the feasibility of the proposed method indicated positive results. References Kosuke Sasaki, Akira Hirata, Tomoo Inoue, Investigation of the illustration generated by crowdsourced microtasks, IPSJ…
- Interface Agent for Pseudo Co-Dining with a Remote Person
“Eating alone” has been one of the serious problems in our society as more elderly people live alone, more people work and live separately, and people’s rhythms of lives become more diverse. Although all the existing research assumed that every participant of social dining had a meal, we found the setting that only a part of the participant has a meal is probable and acceptable for solving the problem.The eating behavior of Surrogate Diner, an interface agent that performs eating behavior as a surrogate for a non-eating participant, is based on the actual eating behavior from recorded videos of co-dining.…
- Twitter Use in the Great East Japan Earthquake
More and more municipal organizations have been using Twitter for information distribution or interactive information exchange. Such official information sources are supposed to be relied upon by citizens when a disaster breaks out, but how such information on social media should be operated is not known. This research investigated the user behavior around the Twitter accounts of municipal organizations in the Great East Japan Earthquake. Through the close investigation, the operation guidelines for a municipal Twitter account were obtained. References Tomoo Inoue, Toward the operation guidelines for municipal social media account: From the analysis of user’s behavior on Twitter in…
- Conversation Analysis Over The Meals
At parties, weddings, dinners, get-togethers, and other events where people gather to socialize, usually they will have a meal together. It is also common in daily life to have conversations over the meal. Why is it that a meal often accompanies conversations? Does communication become better when having a meal together? To clarify these questions, we are investigating how communication differs when having a meal or not. We recorded videos of three friends having a conversation over a meal and another conversation without a meal, then analyzed their speech and gestures in detail. The figure below shows the differences between…
- the Group Future Dining Table (FDT)
Automatic Dish Recommendation System for People Dining Together Dining is clearly indispensable for our life and closely connected to the feeling of happiness and the life quality. So a tabletop dish recommendation system for multiple participants dining together in response to their dining status, which is named the Group Future Dining Table, is presented. The Group FDT system recognizes multiple users’ real time dining activity, stores it as the dining history, and gives dish recommendation in response to the dining status obtained from this information. The recommended dish image is shown on the table by a projector.Cameras are set above…
- Development of a wearable communication recorder(WCR)
It is important to acquire communication data, which is the main research target in communication research. However, there has never been a system that can record incidental communication that occurs regardless of location. It has become possible to record communication data for a lifetime because of the cost reduction of information media and the performance of miniaturized computers. However, most of the data is unnecessary. To get the data of the incidental communication, in this study, we have developed a WCR (wearable communication recorder) which does a little video recording to get necessary data. This system assumes the situation which…
- Task learning MR system
The task learning MR system has various forms besides the self-study in the classroom lecture and the reference book. In order to support the learning of “operation” with the movement of the body, we develop a learning support system by presenting the teaching image in the mixed reality (MR). By using the mixed reality technology, the learner can see the example while doing actual operation by himself. In addition, the system can also identify the status of learners, according to the situation, can show the template, it is an interactive learning support system. System look The figure below shows the…
- Interactive advertising system
Interactive The interactive advertising system is a system that determines the relationship between friends, business, lover, family, and others. By measuring the person’s distance, the number of people and the height by the stereo camera attached to the ceiling, if it is close to the person’s distance, it is identified as a lover, and if there is a person whose height is low, it is distinguished from the family. It is a smart advertising system that automatically shows the advertisement suitable for the group by deriving the relation between each other from the interpersonal distance which is not usually conscious.…
- MR Communication
Mixed reality (MR), as shown in the figure below, is a technology that synthesizes the real world and the virtual world by presenting users the image that superimposes computer graphics (CG) in the real world. Users will feel like the CG appears as it exists in the real world. The distributed meeting can also be realized by superimposing remote meeting participants in the real world as CG avatars using mixed reality. Recently, MR is discussed as a way to let distributed conferences become more realistic. However, how avatars should be displayed in distributed meeting is not discussed yet. While using…
With the internationalization of cooperative activities, opportunities for conversations between native speakers (NSs) and non-native speakers (NNSs) are increasing, but conversations between NSs and NNSs are not always easy. This paper proposes a Speech Speed Awareness System to support conversations between NSs and NNSs by displaying a screen when the speech rate of the NS becomes too fast for the NNS. We analyzed the speech behaviors of NSs and NNSs from a conversation experiment of word-association games conducted to examine the effectiveness of the proposed system. Although no significant differences in the frequency of NNSs asking back to NSs in…
A video communication system “Being Here System (BHS)” provides a feeling of being in the same room with a remote user when conversing. Different from a conventional videoconferencing system that displays the raw image from the remote camera, BHS displays the image that is processed in realtime to match with the local site. It extracts the remote user’s figure from the raw image. It also takes the image of the local site that is hidden by the display. Then it superimposes the user’s figure over the image of the local site, and displays this composite image. We confirmed this method…
With a little help from my native friends: Keyword typing improves mutual understanding in L2 conversation. References Hiromi Hanawa, Xiaoyu Song, Mengyuan Tang, Tomoo Inoue, Speakers’ empowerment with keywords: the speaking competency development for multilingual to contribute in a real-time interaction, Collaboration and Technology, LNCS 10391, pp.148-163, doi>10.1007/978-3-319-63874-4_12, Springer, 2017.8. Hiromi Hanawa, Xiaoyu Song, Tomoo Inoue, Keyword generation by native speaker is quick and useful in conversation between native and non-native speaker, Proceedings of the 2017 IEEE 21th International Conference on Computer Supported Cooperative Work in Design (CSCWD 2017), pp.145-150, 2017.4. Hiromi Hanawa, Xiaoyu Song, Tomoo Inoue, Key-Typing on teleconference:…
We proposed a method to increase the recognition rate of non-native speakers’ speech by inferring non-native speakers’ speech from the native speakers’ speech, which has higher recognition rates.
This research goal is to facilitate real-time serial cooperative work. In real-time serial cooperative work, the order of subtasks is important because a failure of the order leads to a failure of the whole task. So, coordinating the order of workers’ subtasks is necessary to smoothly accomplish the tasks. In this paper, we propose a method that uses vibration to present the action order and coordinate the orders. We conducted a user study to verify the effectiveness of the method in theatrical performance practice, which was an example of real-time serial cooperative work. As a result, order coordination reduced mistakes…
Super-real experience can be realized by combining the objects in both physical and VR environments. We are working on using such kind of super-real experience system as a safety training system. To keep safe, it is necessary to know what the dangers are. But it’s hard to understand them just by listening to lectures. On the other hand, it is also risky to experience the dangers in the real world. Here we use a system that combines objects in both physical and VR environments to provide a safe and realistic way to experience dangers.
Immersive self mental healthcare system has been developed based on a visual counseling method. References Asuki Nakanishi, Yasuhito Noguchi, Toshiya Nakajima, Zhen Yang, Atsuko Matsumoto, Takeshi Kamita, Tsunetsugu Munakata, Tomoo Inoue, A VR self mental healthcare system by SAT-based method, IPSJ SIG Technical Report, Vol.2017-DCC-15, No.35, pp.1-8, 2017.1. Asuki Nakanishi, Yasuhito Noguchi, Toshiya Nakajima, Zhen Yang, Atsuko Matsumoto, Takeshi Kamita, Tsunetsugu Munakata, Tomoo Inoue, Development of a SAT-based self mental healthcare system adopting immersive head-mounted display, VRSJ Research Report, Vol.21, No.CS-4, pp.7-10, 2016.12.
A video communication system “Being Here System (BHS)” provides a feeling of being in the same room with a remote user when conversing. Different from a conventional videoconferencing system that displays the raw image from the remote camera, BHS displays the image that is processed in realtime to match with the local site. It extracts the remote user’s figure from the raw image. It also takes the image of the local site that is hidden by the display. Then it superimposes the user’s figure over the image of the local site, and displays this composite image. We confirmed this method…
In multi-participant conversations, we can hear the voices from multiple directions, which can be helpful with listening comprehension. In the case of remote conversations, we use simple 2-channel stereo acoustic technology to let the user be able to hear voices from multiple directions. References 野口康人, 叶璟, 成合智子, 井上智雄, 多人数遠隔会話のための音像定位分散の効果, 情報処理学会論文誌：デジタルコンテンツ, Vol4, No.1, pp.26-36, 2016.2.
An illustration is a key factor to attract document readers. However, not everybody can make it in good quality. Thus illustration has a high demand. Although conventionally clients request skilled creators to draw illustrations with good rewards, there exist huge needs for less expensive illustrations. We propose a novel method of generating such illustrations by crowd-sourced microtasks. In this method, a basic drawing is generated by unskilled crowd workers from one photograph. Experimentation to study the feasibility of the proposed method indicated positive results. References Kosuke Sasaki, Akira Hirata, Tomoo Inoue, Investigation of the illustration generated by crowdsourced microtasks, IPSJ…
“Eating alone” has been one of the serious problems in our society as more elderly people live alone, more people work and live separately, and people’s rhythms of lives become more diverse. Although all the existing research assumed that every participant of social dining had a meal, we found the setting that only a part of the participant has a meal is probable and acceptable for solving the problem.The eating behavior of Surrogate Diner, an interface agent that performs eating behavior as a surrogate for a non-eating participant, is based on the actual eating behavior from recorded videos of co-dining.…
More and more municipal organizations have been using Twitter for information distribution or interactive information exchange. Such official information sources are supposed to be relied upon by citizens when a disaster breaks out, but how such information on social media should be operated is not known. This research investigated the user behavior around the Twitter accounts of municipal organizations in the Great East Japan Earthquake. Through the close investigation, the operation guidelines for a municipal Twitter account were obtained. References Tomoo Inoue, Toward the operation guidelines for municipal social media account: From the analysis of user’s behavior on Twitter in…
Automatic Dish Recommendation System for People Dining Together Dining is clearly indispensable for our life and closely connected to the feeling of happiness and the life quality. So a tabletop dish recommendation system for multiple participants dining together in response to their dining status, which is named the Group Future Dining Table, is presented. The Group FDT system recognizes multiple users’ real time dining activity, stores it as the dining history, and gives dish recommendation in response to the dining status obtained from this information. The recommended dish image is shown on the table by a projector.Cameras are set above…
It is important to acquire communication data, which is the main research target in communication research. However, there has never been a system that can record incidental communication that occurs regardless of location. It has become possible to record communication data for a lifetime because of the cost reduction of information media and the performance of miniaturized computers. However, most of the data is unnecessary. To get the data of the incidental communication, in this study, we have developed a WCR (wearable communication recorder) which does a little video recording to get necessary data. This system assumes the situation which…
The task learning MR system has various forms besides the self-study in the classroom lecture and the reference book. In order to support the learning of “operation” with the movement of the body, we develop a learning support system by presenting the teaching image in the mixed reality (MR). By using the mixed reality technology, the learner can see the example while doing actual operation by himself. In addition, the system can also identify the status of learners, according to the situation, can show the template, it is an interactive learning support system. System look The figure below shows the…
Interactive The interactive advertising system is a system that determines the relationship between friends, business, lover, family, and others. By measuring the person’s distance, the number of people and the height by the stereo camera attached to the ceiling, if it is close to the person’s distance, it is identified as a lover, and if there is a person whose height is low, it is distinguished from the family. It is a smart advertising system that automatically shows the advertisement suitable for the group by deriving the relation between each other from the interpersonal distance which is not usually conscious.…
Mixed reality (MR), as shown in the figure below, is a technology that synthesizes the real world and the virtual world by presenting users the image that superimposes computer graphics (CG) in the real world. Users will feel like the CG appears as it exists in the real world. The distributed meeting can also be realized by superimposing remote meeting participants in the real world as CG avatars using mixed reality. Recently, MR is discussed as a way to let distributed conferences become more realistic. However, how avatars should be displayed in distributed meeting is not discussed yet. While using…