開発スタッフがアプリ関連の最新技術やニュースをご紹介します。

VR/ARデバイスが小型・軽量化され、MR技術によりヘッドセットをかぶったままでも周囲の様子の確認ができるようになれば、将来はヘッドセットを装着したまま外出する用途もでてきそうです。
そのようなヘッドセットが登場すれば、視野全体をスクリーンとして活用できるため、現状のスマートフォンの小さな画面を置き換えるようなものになるのかもしれません。

外出先や公共の場所でVR/ARデバイスの操作をおこなうには、現状のハンドトラッキングによる手の動きや音声入力による操作の場合、周囲の目が気になります。

周囲が気にならない操作方法として舌による操作がMicrosoft社により研究開発されているようです。舌による操作の場合、口を閉じた状態での操作になるため、周囲から気づかれることなく操作をおこなうことが可能になります。

舌による操作認識の研究には、VRヘッドセットとして「HP Reverb G2 Omnicept Edition」の使用と併用して以下の脳波測定用ヘッドバンド「Muse2」の組み合わせで使用されています。

Muse2
https://www.amazon.co.jp/musu-MU-03-GY-ML-two-import/dp/B07HL2S9JQ

HP Reverb G2 Omnicept Edition
https://jp.ext.hp.com/immersive/reverb_g2_omnicept/

このような研究はVR/ARデバイスの小型・軽量化がさらにすすんで、メガネのように気軽に装着して外出できるようになる時代がくれば、さらに注目されると思います。

(参考)「前歯を舌でタップ」「舌をかむ」　VRヘッドセットを“舌操作”　米Microsoftが開発
https://www.itmedia.co.jp/news/articles/2310/25/news042.html

木曜日担当：nishida

Tweet

Microsoft Azure AI has unveiled a groundbreaking innovation in the realm of image development. They’ve introduced Idea2Img, a multimodal AI framework designed to simplify the process of transforming abstract concepts into tangible images, reducing the need for manual effort.

Idea2Img leverages the power of large multimodal models (LMMs) like GPT-4V to enable a self-refinement process. This iterative approach involves GPT-4V performing prompt generation, selecting draft images, and reflecting on feedback to continually improve results.

What sets Idea2Img apart is its integrated memory module, which tracks the history of exploration for each type of prompt, whether it’s a picture, text, or feedback. This constant interaction between the processes driven by GPT-4V is the key to Idea2Img’s impressive capabilities.

In practical scenarios involving intertwined picture-text sequences, visual design elements, and complex usage descriptions, Idea2Img excels. It can even extract intricate visual information from input images. To assess its effectiveness, the research team conducted user preference studies, comparing Idea2Img with various other models. The results were striking, with a remarkable 26.9% improvement when Idea2Img was paired with SDXL, underscoring its outstanding efficacy in the field.

In conclusion, Microsoft’s Idea2Img is a significant advancement in image development and design. By harnessing the potential of LMMs and iterative self-refinement, it promises to revolutionize the way we create visual assets from abstract ideas. Its adaptability in complex multimodal scenarios and substantial improvements in user preferences make it a game-changing innovation with far-reaching implications for businesses and industries reliant on image creation and design. It has the potential to enhance efficiency and output quality, ultimately leading to greater competitiveness and customer satisfaction.

Asahi

Tweet

Google is making a significant move in the language learning space with a new feature in Google Search that aims to enhance users’ English speaking skills. Initially, this feature is rolling out to Android users in Argentina, Colombia, India, Indonesia, Mexico, and Venezuela, with plans to expand to more countries and languages in the future. This new tool offers interactive speaking practice and personalized feedback for learners translating to or from English, making it a valuable addition to Google’s language learning resources.

The personalized nature of this feature is a standout aspect. Google’s approach includes providing semantic feedback to assess the relevance and comprehensibility of a learner’s response to a given question. Additionally, it identifies areas where grammar improvements can be made and offers example answers at different language complexity levels. During practice sessions, users can also access contextual translations for any words they don’t understand, creating a holistic learning experience.

To develop this feature, Google invested heavily in AI and machine learning. The Google Translate team created the Deep Aligner model for suggesting translations, and other research groups adapted grammar correction models for speech transcriptions, especially for users with accented speech. Google Research teams designed models for semantic feedback and sentence complexity estimation. To ensure a well-rounded language learning experience, Google collaborated with linguists, teachers, and ESL/EFL pedagogical experts, who contributed a mix of human-expert content, AI-assisted content, and in-house human-reviewed material.

While Google’s precise intentions with this feature remain unclear, it has the potential to boost user engagement. Although the blog post doesn’t explicitly indicate that Google is targeting established language learning apps like Duolingo, it is an intriguing move in a field with substantial profit potential. Google has previously ventured into language learning and education tools, and the success and direction of these efforts may depend on user adoption and popularity.

You can also check out the blog from Google here.

Yuuma

Tweet

Microsoft has revealed its plan to phase out NT LAN Manager (NTLM) authentication in Windows 11 to enhance security and focus on bolstering the Kerberos authentication protocol. This move includes the introduction of features like Initial and Pass Through Authentication Using Kerberos (IAKerb) and a local Key Distribution Center (KDC) for Kerberos in Windows 11.

NTLM, a security protocol from the 1990s, was originally designed for user authentication, integrity, and confidentiality. However, it has been replaced by Kerberos since Windows 2000, though it continues to be used as a fallback option. NTLM relies on a three-way handshake for user authentication and uses password hashing, while Kerberos employs a two-part process with encryption.

NTLM has been found to have inherent security weaknesses and is vulnerable to relay attacks, which could potentially allow unauthorized access to network resources.

Microsoft is actively working to address hard-coded NTLM instances in its components as part of its preparation to disable NTLM in Windows 11. These changes will be enabled by default, with no need for additional configuration in most scenarios. NTLM will still be available as a fallback for maintaining compatibility with existing systems.

Asahi

Tweet

Google is introducing generative AI capabilities to its popular Google Photos app through the release of the Pixel 8 and Pixel 8 Pro smartphones. Initially revealed at Google’s I/O developer conference in May, this feature enables more advanced photo edits, such as filling in gaps, repositioning subjects, and adjusting the foreground or background of images. Previously, achieving these effects required external tools like Google’s Magic Eraser or professional software like Photoshop, involving more manual effort.

Using generative AI, Google Photos allows users to perform complex edits like resizing or repositioning subjects with ease. Users can tap on the object they wish to edit, drag it to move or pinch to resize, and make contextual adjustments to lighting and background. Magic Editor also offers multiple output options for user preference. However, Google acknowledges that the feature is in its early stages and may not always produce the desired results but hopes to improve it with user feedback and technological advancements.

The real-world testing of Magic Editor is imminent, and Google Photos users, especially those on the newer Pixel devices, will have the opportunity to try it out. With 1.7 billion photo edits made by Google Photos users monthly, the potential for learning and improvement is significant. This feature is part of a broader set of AI-powered photo-editing tools for the Pixel 8 and 8 Pro, including Best Take, Zoom Enhance, and enhancements to Magic Eraser, which will be available on Pixel 8 devices starting October 12.

Yuuma

Tweet