Kategorie: Allgemein

Screenshot of the embodied avatar system “Diana.“ Credit:
Brandeis University

Imagine a classroom in the future where teachers are working
alongside artificial intelligence partners to ensure no student
gets left behind.

This is part of a larger vision of future classrooms where human
instruction and AI technology interact to improve educational
environments and the learning experience.

James Pustejovsky, the TJX Feldberg Professor of Computer
Science, is working towards that vision with a team led by the
University of Colorado Boulder, as part of the new $20 million
National Science Foundation-funded AI Institute for Student-AI
Teaming.

The research will play a critical role in helping ensure the AI
agent is a natural partner in the classroom, with
language and vision capabilities, allowing it to not only hear what
the teacher and each student is saying, but also notice gestures
(pointing, shrugs, shaking a head), eye gaze, and facial
expressions (student attitudes and emotions).

Pustejovsky took some time to answer questions from BrandeisNOW
about his research.

How does your research help build this classroom of the
future?

For the past five years, we have been working to create a
multimodal embodied avatar system, called „Diana,“ that interacts
with a human to perform various tasks. She can talk, listen, see,
and respond to language and gesture from her human partner, and
then perform actions in a 3-D simulation environment called
VoxWorld. This is work we have been conducting with our
collaborators at Colorado State University, led by Ross Beveridge
in their vision lab. We are working together again (CSU and
Brandeis) to help bring this kind of „embodied human computer
interaction“ into the classroom. Nikhil Krishnaswamy, my former
Ph.D. student and co-developer
of Diana, has joined CSU as part of their team.

How does it work in the context of a classroom
setting?

At first it’s disembodied, a virtual
presence on an iPad, for example, where it is able to recognize
the voices of different students. So imagine a classroom: Six to 10
children in grade school. The initial goal in the first year is to
have the AI partner passively following the different students, in
the way they’re talking and interacting, and then eventually the
partner will learn to intervene to make sure that everyone is
equitably represented and participating in the classroom.

Are there other settings that Diana would be useful in
besides a classroom?

Let’s say I’ve got a Julia Child app on my iPad and I want her
to help me make bread. If I start the program on the iPad, the
Julia Child avatar would be able to understand my speech. If I have
my camera set up, the program allows me to be completely embedded
and embodied in a virtual space with her so that she can help
me.

How does she help you?

She would look at my table and say, „Okay, do you have
everything you need.“ And then I’d say, „I think so.“ So the camera
will be on, and if you had all your baking materials laid out on
your table, she would scan the table. She’d say, I see flour,
yeast, salt, and water, but I don’t see any utensils: you’re going
to need a cup, you’re going to need a teaspoon. After you had
everything you needed, she would tell you to put the flour in „that
bowl over there.“ And then she’d show you how to mix it.

Is that where Diana comes in?

Yes, Diana is basically becoming an „embodied presence“ in
the human-computer
interaction: she can see what you’re doing, you can see what
she’s doing. In a classroom interaction, Diana could help with
guiding students through lesson plans, through dialog and gesture,
while also monitoring the students‘ progress, mood, and levels of
satisfaction or frustration.

Does Diana have any uses in virtual learning in
education?

Using an AI partner for virtual learning could be a fairly
natural interaction. In fact, with a platform such as Zoom, many of
the computational issues are actually easier since voice and video
tracks of different speakers have already been segmented and
identified. Furthermore, in a Hollywood Squares display of all the
students, a virtual AI partner may not seem as unnatural, and Diana
might more easily integrate with the students online.

What stage is the research at now?

Within the context of the CU Boulder-led AI Institute, the
research has just started. It’s a five-year project, and it’s
getting off the ground. This is exciting new research that is
starting to answer questions about using our avatar and agent
technology with students in the classroom.

Link to original article

Originally published by Tessa Venell, Brandeis University 

November 20th 2020

Cloud-enabled Al and machine learning are providing healthcare
stakeholders with the tools needed for a faster and smarter
approach to combatting the COVID-19 virus.
(WrightStudios/Shutterstock)

As COVID-19 began spreading across the U.S., healthcare
organizations were forced to quickly reassess their technology, and
pull future plans for digital transformation forward.

In record time, many organizations overhauled legacy systems to
better manage and care for the uptick in patient visits, while
safely storing data to ensure efficiency as the pandemic
evolved.

One of the most pressing priorities for healthcare organizations
was expediting their adoption of cloud technologies to more
efficiently manage the deluge of patient information, ensure
streamlined workplace practices and enable information sharing with
greater ease. As local leaders made decisions about how to keep
their populations safe, cloud infrastructure provided the ability
to collect, analyze, and share data securely across and among a
global network of organizations.

Through this period of rapid cloud adoption, there has also been
a swift uptick in the use of artificial intelligence (AI) and
machine learning technologies. From enabling information sharing
and analysis without sacrificing data privacy, to ensuring patients
with the most urgent needs are given the quickest response, these
technologies have revolutionized the COVID-19 healthcare response
and will remain critical well beyond the pandemic.

Here are just a few of the ways in which COVID-19 has spurred
lasting digital transformation within the healthcare industry:

De-identification of patient data

With machine learning capabilities, healthcare organizations are
better equipped to ensure the privacy of patient data, making it
easier to aggregate data across multiple sources and garner helpful
insights about the COVID-19 virus. De-identification, the process
of removing identifying information from patient data, is critical
to the sharing of health information with non-privileged parties
for research purposes, the creation of datasets from multiple
sources for analysis, and anonymizing data so it can be used in
advanced analytics and machine learning models.

As an example, the Google Cloud Healthcare API can detect
sensitive data, such as protected health information (PHI), and
mask, delete, or otherwise obscure it.

To enable researchers to study critical COVID-19 information for
fighting the virus, patient identities from DICOM assets, such as
lung x-rays, can be removed at scale using the same type of machine
learning technology that scans YouTube for copyright infringement,
making the data usable for analytics in high-definition. Further,
testing data can be de-identified, accelerating discovery. When
properly hashed, such data can then be safely re-identified
allowing researchers to more effectively recruit for public health
programs like clinical trials.

Natural language processing for call center responses

All types of public health organizations today are inundated
with more patient requests than ever beforeand many were not
initially equipped to manage this increase.

With cloud-based AI and machine learning models, however,
organizations can build the call center of the future. Using
natural language processing and sentiment analysis, healthcare
providers can automatically prioritize calls based on need.

This technology allows an organization to optimize its approach
to answering/prioritizing inquiries based on everything from the
distress of the voice to the age of the voice. And while they’re
smart, many of these APIs are engineered with privacy in mind. They
don’t store private data, helping ensure patient
confidentiality.

Supply chain decisions informed by predictive analytics

Cloud isn’t just supporting healthcare organizations through
research and treatment decisions. It is also helping them get ahead
of supply shortages at a time when equipment is more critical to
survival than ever before.

As organizations look to provide critical healthcare equipment
such as PPE and ventilators to those in need, cloud’s predictive
analytics can help those managing the supply chain better
understand where shortages exist, and where they will soon be, in
order to allocate before there is an issue.    

Matching algorithms are easily implemented alongside predictive
services to reduce waste in the supply chain, enabling real-time
visibility to both suppliers and procurers.

Cloud-enabled Al and machine learning are providing healthcare
stakeholders with the tools needed for a faster and smarter
approach to combatting the COVID-19 virus. While the mission today
is singular, this technology, along with the innovative ideas
coming from our nation’s top minds, will change the face of
healthcare as we know it, allowing for a greater patient experience
than ever before.

Originally published by
Lisa
Noon, Deloitte | Nov 23, 2020
Fierce
Healthcare

ADELPHI, Md.— Researchers at the U.S.
Army’s corporate research laboratory developed an artificial
intelligence architecture that can learn and understand complex
events, enhancing the trust and coordination between human and
machine needed to successfully complete battlefield missions.

The overall effort, worked in collaboration with the University of California, Los
Angeles and Cardiff
University, and funded by the laboratory’s Distributed
Analytics and Information Science International Technology
Alliances, addresses the challenge of sharing relevant knowledge
between coalition partners about complex events using
neuro-symbolic artificial intelligence.

Complex events are compositions of primitive activities
connected by known spatial and temporal relationships, said U.S.
Army Combat Capabilities Development Command, now referred to as
DEVCOM, Army Research
Laboratory researcher Dr.
Lance Kaplan. For such events, he said, the training data
available for machine learning is typically sparse.

To further understand complex events, imagine people in a crowd
taking pictures of an iconic government building. The act of
picture taking involves primitive events/actions. Now, imagine that
some of the people are coordinating their picture taking for the
purpose of a reconnaissance mission. A certain sequence of
primitive events such as picture taking occurs. Clearly, it would
be good for a force protection system to detect and identify these
complex events without generating too many false alarms due to
random primitive events acting as clutter, Kaplan said.

This new neuro-symbolic architecture enables injection of human
knowledge through symbolic rules (i.e. tellability), while
leveraging the power of deep learning to discriminate between the
different primitive activities.

This is accomplished following a neuro-symbolic architecture
where the lower layer is composed of neural networks that are
connected through a logical layer to form the complex event
classification decision, Kaplan said. The symbolic layer
incorporates known rules that enable learning the lower layers
without having to train labeled data for the primitive
activities.

Two different approaches have been developed to enable learning
at the neural layers by propagating gradients through the logic
layer.

The first, Neuroplex, uses a neural surrogate for the symbolic
layer. Second, DeepProbCEP, uses DeepProbLog to propagate the
gradients.

Neuroplex was evaluated against pure deep learning methods over
three types of complex events formed by a sequence of images, a
sequence of sound clips and a nursing activity data set collected
from motion capture, meditag and accelerometer sensors.

The experiments and evaluation showed that Neuroplex is capable
of learning to efficiently and effectively detect complex events,
which cannot be handled by state-of-the-art neural network
models.

During the training, Neuroplex not only reduced data annotation
requirements by one hundred times, but also significantly sped up
the learning process for complex event detection by four times.

Similarly, experiments on urban sound clips demonstrated over a
two times improvement in complex event accuracy for DeepProbCEP
against a two-stage neural network architecture.

“This research demonstrates the potential for
neuro-symbolic artificial intelligence architecture to learn how to
distinguish complex events with limited training samples,” Kaplan
said. “Furthermore, it is demonstrated that the systems can learn
primitive activities without the need for annotations of the simple
activities.”

In practice, he said, the initial layers of the neuro-symbolic
are pre-trained, but the amount of data to collect and label can be
greatly reduced, lowering costs.

In addition, the neuro-symbolic architecture can leverage the
symbolic rules to update the neural layers using a small set of
labeled complex activities in situations where the raw data
distribution has changed. For example, in the field it is raining,
but the neural network models were trained over data collected in
sunny weather.

The neuro-symbolic learning is also able to incorporate the
superior pattern recognition capabilities of deep learning with
high level symbolic reasoning.

“This means the AI system can naturally provide explanations
of its recommendations in a human understandable form,” Kaplan
said. “Ultimately, this will enable better trust and coordination
with the AI agent and human decision makers.”

The symbolic layer also enables tellability. In other words, he
said, the decision maker can define and update the rules that
connect primitive activities to complex events, both at the
initialization stage, where a perception module is untrained, as
well as during the fine-tuning stage, where a perception module is
a pre-trained off-the-shelf model and needs to be fine-tuned to a
specific environment.

“This research is able to leverage the advancements
in deep learning and symbolic reasoning,” Kaplan said. “The
dimensionality of the sensor data whether it is time-series data
(e.g., accelerometers) or video is infeasible for pure symbolic
reasoning. Similarly, deep learning is unable to learn patterns
that manifest over large time and spatial scales inherent in
complex events. The hybrid of neuro-symbolic learning is necessary
for complex event processing.”

The research directly supports the Army Priority Research Area
of artificial intelligence by increasing speed and agility in which
we respond to emerging threats, Kaplan said. Specifically, the
research is focused on AI for detecting and classifying complex
events using limited training data to adapt to changing
environments and learn emerging events.

The work also supports network command, control, communications
and intelligence by engendering trust between AI and human agents
representing different coalition partners by the AI incorporating
symbolic explanations and the human agents using symbolic rules to
guide the reasoning of the AI, he said.

“Complex event processing is difficult, and the
research is still in its infancy,” Kaplan said. “Nevertheless,
we have made great advancements over the last two years to develop
the neuro-symbolic framework. The research still needs to consider
more relevant data sets, and there are still other questions to
answer regarding the human-machine interface. In the long term, I
am confident that a neuro-symbolic AI will be available to Soldiers
to provide superior situational awareness than the available to the
adversary.”

Originally published by

U.S. Army DEVCOM Army Research Laboratory Public Affairs | 
November 17, 2020

This research was recently presented at the virtual International Conference on Logic
Programming, and will be featured during the upcoming ACM Conference on Embedded Networked
Sensor Systems, or SenSys 2020, scheduled virtually for Nov.
16-19.