Cross-Talks Workshop - Program

23 junho 2024, 19:15 • Ana Almeida Matos

List of talks (order may be adjusted according to unforeseen constraints)


24 of June, 10:00-12:30 - Alameda @ room F2, or Zoom (course link)


Title: Co-designing Playful Urban Public Space Technologies with and for Neurodiversity
By: Patricia Piedade
Abstract: Access to public spaces is of the utmost importance for social cohesion, inclusion, and civic engagement. Nevertheless, these spaces remain incredibly uncomfortable environments for neurodivergent individuals. My proposal aims to leverage growing trends in Playable City technology design to promote a sense of belonging for neurodivergent individuals in urban public spaces. We’ll take a Participatory Action Research approach, collaborating with neurodiverse (i.e. neurodivergent and neurotypical) stakeholders to design said technologies. We will establish a joint knowledge base of current issues, co-design a technological solution to address them and evaluate its impact on neurodiverse stakeholders. Through this process, we will gain a better understanding of the challenges, opportunities, and design solutions within this space. Furthermore, we will empower local communities and contribute to local policymaking. We aim to make positive and long-lasting change, demonstrating the potential of co-designed playful urban technologies for inclusive urban spaces that embrace neurodiversity.


Title: Environmental Impact of CI/CD Pipelines in Open-Source Projects
By: Nuno Saavedra
Abstract:
The adoption of Continuous Integration and Continuous Deployment (CI/CD) pipelines has revolutionized software development, enabling rapid iteration and delivery of high-quality code. Nonetheless, these benefits come with a frequently neglected cost: the environmental impact of CI/CD runs. This talk delves into the carbon footprint produced by CI/CD pipeline runs in open-source projects. Specifically, we will examine the carbon footprint of open-source projects using GitHub Actions, the most popular CI/CD platform today.
With the goal of reducing the environmental impact of CI/CD, we will explore methods to identify wasted computational power. Then, we will delve into potential strategies to mitigate the identified issues. Finally, we will talk about creating tools that automatically apply these strategies to CI/CD scripts. This talk aims to foster awareness and encourage the adoption of sustainable practices when using CI/CD platforms, ultimately contributing to greener software development.


Title: Optimizing Symbolic Execution with Learning-Enhanced Portfolio Solvers and Encodings
By: Filipe Marques
Abstract:
Symbolic execution is a program analysis technique used for testing and validating software. Instead of using specific constants as input, symbols are supplied, and SMT solvers are used to reason about these symbols. In large systems, symbolic execution can be slowed down due to complex formulas.
To address this problem, one can use a portfolio approach, where multiple SMT solvers are called in parallel, and the answer from the solver that responds first is used. This is effective because different solvers excel at different types of formulas.
In this work, we not only use multiple solvers but also encode constraints in different theories (e.g., Bitvectors vs. Integers). By leveraging machine learning, one can optimize this process by learning which encoding and solver to use for each formula during symbolic execution.


Title: Towards Net Zero: Using an Electric Vehicle to Manage Household Energy
By: Manuel Pereira
Abstract: As the world transitions toward net-zero emissions, innovative energy management strategies are essential. This presentation explores a novel approach that leverages electric vehicles (EVs) to optimize household energy consumption. By integrating EVs, Battery Energy Storage Systems (BESSs), and renewable energy sources, we aim to minimize costs, enhance grid reliability, and reduce environmental impact. This presentation uses a Mixed Integer Linear Programming optimization model. This approach mitigates resource waste associated with renewable energy sources like photovoltaic (PV) and wind. With EVs, excess energy can be stored and utilized efficiently, virtually eliminating waste.


Title: Day-ahead MILP Optimization Model for Energy Communities Management using Load Shifting, EVs and V2G Technology
By: Nuno Velosa
Abstract: The correct use of renewable energy sources is important to achieve decarbonization in the energy sector. Energy communities play a crucial role in this matter. Nevertheless, the proper balance of supply and demand is not easy when the generation is completely dependent on weather conditions. When the EC demand is too high, the renewables can be insufficient to cover all the consumption needs, meaning the need to buy electricity from the grid, whereas when the EC production is too high, without the proper use of the energy storage systems, it will not be used by the members. These limitations and challenges prevent to bring renewable energy communities into reality, leading to the need of developing strategies to maximize the use of the renewable resources as well as minimize the associated costs. Load Shifting techniques have been proved that can contribute to overcome these challenges. This work presents a day-ahead load shifting strategy for renewable energy communities combined with electric vehicles, vehicle-to-grid technology and time-of-use electricity rates, considering the members' flexibility, which is used for shifting the appliances usage over the day according to the community goals. The results demonstrated that much better results can be achieved in just a single day without too much effort.


Title: EcoAcoustics: Real-Time Monitoring through Acoustic Detection of Riverine Fish Species
By: Ziqi Huang
Abstract: Understanding and preserving riverine biodiversity is essential for ecosystem health. We introduces a novel acoustic monitoring system that utilizes underwater sensors to identify fish species based on their unique sounds. This technology not only automates the collection and analysis of bioacoustic data but also engages the public in real-time by notifying them of nearby fish species through a mobile application. The main challenges include handling imbalanced data with prevalent fish sounds and managing multi-label issues where recordings capture multiple species simultaneously. Manual data labeling by biologists is required, making the process time-intensive. We aims to advance scientific knowledge, enhance community engagement, and promote environmental stewardship. By leveraging machine learning and sensor technology, we provide a scalable model for ecological monitoring and public education, fostering a deeper connection between communities and their local environments. Future efforts will focus on refining our algorithms and expanding user interaction to foster greater community involvement in conservation practices.


Title: Enhancing Document Understanding Through Large Vision-Language Models
By: Helder Dias
Abstract: Vision-Language Models (VLM) have recently achieved remarkable success and public notice, and are becoming one of the most attractive research artifacts. Applications needing document understanding can leverage these new multimodal capabilities to improve many tasks, ranging from information retrieval, question answering over forms, reporting financial data, research measurements, etc. These tasks have been solved mainly as a token classification problem, but the simple and naive textual processing cannot represent colors and include spatial dependencies. Vision-language models are typically limited to processing low resolution images.
This research addresses the challenge of incorporating high-resolution images supporting the analysis of small text font sizes, while maintaining computational efficiency. Recently, new strategies of dividing the image and using additional representations have been proposed that serve as first exploration ideas for further developments.


Title: Making Extended Realities Accessible Using Diverse Modalities To Increase Sense Of Being There and Embodiment
By: Noha Mokhtar
Abstract: Extended Realities are increasingly making their way into our lives due to mainstream technology's dependence solely on controllers and hand interactions, automatically excluding users with limited mobilities. With various innovative interaction techniques, we try to reach that gap by creating interactions dependent on other modalities such as eye-gaze interaction and other multi-modal interactions, meanwhile trying to increase the sense of presence and embodiment within these Virtual environments.
So far, we created an interaction technique with 6 hand representations and 4 animations and it proved to have a significant difference in regards to presence and self-location an embodiment factor with 2 animations: Dwell (1.5 seconds) and Snap (0.5 seconds).


Title: Enhancing Early Detection of Colorectal Cancer through Microbiome Analysis
By: André Salgado
Abstract: Colorectal cancer (CRC) is a major health issue worldwide, and current diagnostic methods often involve invasive procedures like colonoscopies, which can be uncomfortable and have limited accessibility. This talk introduces a pioneering approach to diagnose CRC earlier and more accurately by analyzing the gut microbiome – the community of microorganisms living in our intestines.
The primary objective of this research is to develop a new diagnostic tool that uses gut microbiome biomarkers to detect CRC in its early stages. To achieve this, the first step is to establish a sophisticated metagenomics analysis pipeline to identify reliable biomarkers within the gut microbiome.
A key focus of the talk will be on the application of advanced machine learning techniques to create classifiers capable of distinguishing between healthy and cancerous microbiome profiles based on the identified biomarkers. These machine learning models are trained to recognize intricate patterns and associations unique to each individual, thereby enhancing the accuracy of CRC detection.
The talk will also cover the innovative use of federated learning architectures, a method that can enhance the reliability of CRC detection by allowing data analysis across multiple, large, and diverse datasets without compromising privacy. This approach aims to make the diagnostic tool robust and universally applicable.

Title: Bridging AI and Ethics - Ensuring Copyright Compliance in LLM Training Data
By: André Duarte
Abstract: How can we detect if copyrighted content was used in the training process of a language model, considering that the training data is typically undisclosed? We are motivated by the premise that a language model is likely to identify verbatim excerpts from its training text. We propose DE-COP, a method to determine whether a piece of copyrighted content was included in training. DE-COP's core approach is to probe an LLM with multiple-choice questions, whose options include both verbatim text and their paraphrases. We construct BookTection, a benchmark with excerpts from 165 books published prior and subsequent to a model's training cutoff, along with their paraphrases. DE-COP's preliminary results look promising and worth investigating.



26 of June, 15:00-17:00 - Tagus @ room 0.13, or Zoom (course link)

Title: Evaluation Metrics for Vision-Language Models
By: Gonçalo Gomes
Abstract: The evaluation of vision-language models is far behind compared to language model tasks, such as machine translation, necessitating advancements in assessment methodologies. Vision-language tasks like image captioning suffer from misinterpretations of fine-grained details, resulting in errors such as object hallucinations or flawed compositional reasoning in generated captions. My proposal research aims to develop nuanced evaluation metrics targeting the shortcomings of existing methods, to rectify model hallucinations and compositional deficits. The intricate nature of multimodal tasks also inherently encompasses uncertainties, requiring the exploration of methodologies to quantify them. This need becomes particularly pronounced when evaluating vision-language models, particularly in tasks such as generating captions from images and vice versa, where the metrics for automatically evaluating these models deliberately seek to ensure fairness and mitigate biases rooted in pre-trained models.


Title: Integrating Graphical Representations into ITLingo ASL for Enhanced Software Application Specifications
By: Tiago Marcelino
Abstract: Developed by an IST community, ITLingo ASL (Application Specification Language) stands as a controlled natural language (CNL) purpose-built for the rigorous specification of software applications in a platform-independent and simplified manner. ASL specifications have predominantly relied on textual syntaxes and have proven advantageous in fostering clarity and independence in specifying various applications, such as RPA projects, BI solutions, and low-code web applications, with implementations leveraging popular Xtext and Langium frameworks. However, as software systems evolve in complexity and scope, the need arises to explore them using visual models to complement and support textual specifications. This work aims to develop a modelling tool (ITLingo-ASL-Modeller) that incorporates graphical representations into the ASL specification process to enhance usability and clarity. For this project, a flexible meta-modelling framework with configurable graphical representations was selected: the ADOxx framework. The work entails the practical establishment of the core ASL metamodel within the ADOxx framework and a user experiment evaluation of a real-world use case scenario. We seek to uncover the potential advantages and challenges of integrating visual modelling capabilities into ITLingo ASL. By bridging textual and visual representations, we aim to empower software engineers and domain experts with enhanced tools for articulating and comprehending intricate software specifications, ultimately fostering efficiency, clarity and independence in software development projects.


Title: Theory of Mind in Human-AI Interactions
By: António Fernandes
Abstract: The rise of Artificial Intelligence over the past decades has promoted the development of complex hybrid societies composed of humans and machines. To minimize the risk of potentially harmful interactions, AI systems must be equipped with a high degree of social understanding and aligned with societal values, including fairness and transparency. These features require an understanding of others’ mental states, i.e., Theory of Mind (ToM), which has not been effectively modeled despite its relevance. Here, we investigate the impact of ToM on strategic human decision-making through the analysis of a behavioral experiment. Specifically, we explore how humans employ ToM within their reasoning processes and how higher levels of ToM influence prosocial behavior in mixed-motive interactions. For this, we designed an experimental study where participants play one of two instances of the Centipede Game, a sequential two-player game that requires backward reasoning, a process dependent on ToM. This project is situated within a broad and thriving interdisciplinary research area at the intersection of Behavioral Economics, Computer Science, and Psychology.


Title: A serious game learning platform for personalized education for neurodiverse children
By: Joana Brito
Abstract: Neurodevelopmental disorders, affecting 8-15% of children, often impair executive functions — cognitive skills crucial for development, including memory and attention. These children typically lag behind their neurotypical peers. Creating opportunities for children to train these functions is vital, especially for those with developmental weaknesses. Research indicates that serious games are effective for that purpose, as well as have a motivational impact on children. Our project developed a serious game platform for children aged 6 to 12 that provides both assessment and training of executive functions. So far, we have taken this platform to schools with neurodiverse children to collect multimodal data including physiological metrics. We will use this data to develop feedback algorithms tailored to each child’s unique needs. A major interdisciplinary challenge is integrating this multimodal data. Our hybrid approach combines cognitive theory insights to contextualize gameplay behavior, enhancing algorithm precision without over-reliance on sensor data.


Title: Diagnosing Pulmonary Embolism using Deep Learning
By: João Marques
Abstract: Pulmonary embolism (PE) is a life-threatening condition characterized by diagnostic challenges due to the non-specific nature of its clinical presentation. Confirmation typically relies on Computed Tomography Pulmonary Angiography (CTPA), but electrocardiography (ECG) can also provide valuable diagnostic insights. ECG is non-invasive and offers critical information about the patient's condition. Recent studies suggest that some ECG findings may correlate with the severity of PE. Building on this, we are developing a Deep Learning algorithm that leverages both ECG data and CTPA results, aiming not only to predict the likelihood of PE but also to assign a severity score, enhancing diagnostic precision and patient care.


Title: Specification-Driven Generation of Summaries for Symbolic Execution
By: Frederico Ramos
Abstract: Symbolic execution is a popular program analysis technique that has been successfully used for bug-finding and bounded verification in various modern programming languages. Despite its popularity, symbolic execution still suffers from two main limitations when applied to real-world code: interactions with the runtime environment and path explosion. Symbolic summaries are the standard solution to tackle these challenges. However, the development of summaries remains to this day a manual task that is both time-consuming and error-prone. To address this, we propose SumGen, a new tool for automatically generating summaries from Separation Logic specifications.


Title: Leaning-based VR Locomotion for Games
By: Adrian Leon
Abstract: Towards the goal of promoting a more inclusive access for extended reality communities, here we pretend to analyze the interplay between use of a Leaning-based system in VR. For this, we use the head and position tracking data of the device itself that transforms the user's head-torso into a joystick similar movement. Besides, the addition gaze data to dig into how locomotion through this system affects the user's focus onto an object. This interplay is further analyzed and served as the foundation for a following combination of locomotion, pointing and selection systems in VR.


Title: Ergonomic Functional Extensionality.
By: Henrique Guerra
Abstract: Proof assistants based on type theory allow us to state and prove the equality of terms that reduce to the same normal form. Thus, they don’t support functional extensionality, i.e., one cannot prove the equality of two functions that are pointwise equal when they have different normal forms. Existing approaches to add functional extensionality only allow the elimination of equality into proof-irrelevant types or do not reduce in several terms (even closed
terms). We find new reduction rules to get normalization, using different techniques, such as program synthesis, and improve the type checker so that terms that do not reduce are ruled out.

Title: Statistical Learning for Multi-Omics Integration in Precision Cancer Medicine
By: Rita Baião
Abstract: Precision cancer medicine approaches have shown great promise in providing tailored and effective treatments. Consequently, cancer cells have been extensively molecularly and phenotypically characterized generating large, heterogeneous, and complex datasets. However, this diversity and high-dimensionality pose methodological and computational challenges that remain largely unaddressed. Recent advances in machine learning showed an enormous potential to efficiently tackle high-dimensional datasets and develop models to represent and integrate diverse types of features. This project harnesses recent developments in deep learning to develop a machine learning framework that integrates existing and emerging large-scale cancer multi-omics datasets. The main goal is to design and explore novel neural network models, including variational autoencoders, to integrate these datasets and systematically benchmark them against state-of-the-art approaches by defining robust validation procedures, which are currently lacking. Contrasting existing approaches, this work will integrate multi-omic data from both preclinical (cell lines) and clinically relevant (organoids and tumor biopsies) cancer samples, enabling predictions of treatment outcomes for patients and prioritizing the most effective drug treatments for clinical validation.