Ph.D. Student, Artificial Intelligence of Things

Eugene Sy施辰穎

I work on machine learning for speech, health and finance: screening for cognitive impairment from recorded memory tests, tracing information channels in financial networks, and running language models on edge devices.

National Taiwan University Academia Sinica

Taiwan International Graduate Program (TIGP-AIoT), a joint Ph.D. program of Academia Sinica and National Taiwan University

Selected researchDownload CV

Eugene Sy smiling outdoors, holding a coffee cup
6peer-reviewed papers, plus one journal manuscript under review
1stplace in argument unit identification, NTCIR-17 FinArg-1
2–3 send-to-end response time on an edge device, down from 7–10 s
4.0GPA, Class Valedictorian, Master's in Data Science (TMU)

Selected research

Research and projects.

GitHub

Clinical AI · First author

PRISM-CI

Screening for cognitive impairment from a recorded memory test.

PRISM-CI combines acoustic, facial and audio-visual features from older adults recalling a word list (the CVLT-II), with no transcript needed. It was evaluated on 276 community-dwelling Mandarin-speaking older adults and compared with four published methods on the same data.

0.761held-out AUROC
276participants
3feature branches: acoustic, visual, audio-visual

Under review · IEEE Journal of Biomedical and Health Informatics

Graph learning · Co-first author

Detecting information channels in U.S. congressional trading.

GAP-TGN is a temporal graph network that links congressional stock trades to lobbying, campaign finance and committee data, and keeps its node memory up to date during the months before a trade's outcome is known. The paper also introduces Capitol Gains, a point-in-time dataset covering 2013–2025.

ICLR 2026 Workshop on Advances in Financial AI

arXivOpenReview

GAP-TGN architecture: a trade event is encoded, node memory is retrieved with graph attention, fused with market signals to predict the outcome, and memory is updated by a message and GRU after the prediction
The GAP-TGN architecture, from Roodman, Sy et al. (2026)

Edge AI · Industry internship

On-device language assistant.

At AUO Mobility Solutions, I replaced a rule-based command dispatcher with an LLM tool-calling backend for an edge hardware product, added open-ended Q&A, and kept English–Chinese code-switched input working.

7–10 s → 2–3 send-to-end response time

KV-cache reuse · speculative decoding · flash attention

Financial NLP

Argument mining in financial text.

Ensembles of BERT-family models that identify arguments in financial text. The approach placed 1st in argument unit identification at NTCIR-17 FinArg-1, and the 2025 journal version reached 77.08% macro-F1 on that task.

PaperCode

Ensemble architecture: several BERT models and BART, plus an MLP over sentence-pair scores, feed a majority vote that labels claims and premises, and support or attack relations
The ensemble, from Sy et al., ROCLING 2023

More projects

Course and side projects.

PathSense study design: three groups play the same tilt game, with no feedback, sound feedback or vibration feedback as the ball nears the wall
PathSense: same tilt game, three feedback conditions
Sonic Lockpicker screen: a Wordle-style grid of number guesses with green and yellow hints
Sonic Lockpicker, an eyes-free audio game
Fitts’s law plot: reaction time rises much faster with difficulty for head control than for mouse control
Fitts’s law: head-controlled pointing vs. a mouse
Spring 2026 · Multimodal Human-Computer Interaction (CSIE7641), NTU, taught by Prof. Shan-Yuan Teng

PathSense and multimodal interaction studies

Final project: PathSense. A smartphone tightrope game: players tilt an Android phone in their non-dominant hand to keep a ball on a narrow path, with haptic vibration, audio tones or no feedback as the ball nears the edge. With 25 participants across four difficulty levels, the feedback type made no significant difference (Kruskal-Wallis p > 0.19), but a tenfold gap in baseline skill showed that individual differences mattered more than modality. Built with Phaser 3, Node.js and Socket.IO, and written up as an ACM-format paper.

Mini-projects. Sonic Lockpicker, an audio-only, eyes-free lockpicking game with spatial audio and voice input; a gaze- and expression-driven TikTok interface; and Tactile Theater, ambient video choreographed with a robotic arm for touch.

Weekly experiments. Auditory just-noticeable-difference tests, offline speech recognition, a head-controlled mouse with a Fitts’s law pointing study, webcam gaze tracking, and haptic emoji recognition with a robotic arm.

Course pageGitHubTactile Theater demo
Architecture: ad text is processed, searched against a FAISS vector store of legal documents, checked against violation patterns, and judged legal or illegal by an LLM June 2025 · Artificial Intelligence, NTU

Spotting illegal health-food ads

A retrieval-augmented system that flags food and supplement ad copy for violations of Taiwan's Food Safety and Sanitation Act: a vector store of regulations and past violation cases, a rule-based risk score, and an LLM for the final call. Built by a team of seven; I made the slides and presented the project. The final system scored an F1 of 0.867, eighth of 20-plus teams in the class competition.

Demo video
Spring 2025 · Artificial Intelligence (CSIE5400), NTU

Captioning, RAG and a multi-agent system

HW1. Evaluated the BLIP and Phi-4 captioning models on MSCOCO and Flickr30k (BLEU, ROUGE, METEOR), then built multimodal-LLM style transfer, with Phi-4 writing prompts for Stable Diffusion 3 (text-to-image) and Stable Diffusion 1.5 (image-to-image).

HW2: RAG. First a question-answering system over my own CV with Phi-2: without retrieval the model invented a biography (a Hong Kong politician born in 1882); with retrieval every claim traced back to the CV. Then a page-retrieval pipeline for long PDFs: native text plus Phi-4 OCR, chunks at three sizes, BGE dense embeddings fused 70/30 with BM25 keyword search, a re-ranker, and Phi-4 choosing the page.

HW3: multi-agent system. Four AutoGen agents rate a restaurant from its reviews: one fetches the reviews, one maps them to a fixed list of food and service adjectives, one turns those into scores with a deterministic lookup (so the LLM never does arithmetic), and one combines them with a geometric mean. It passed 4 of 5 public test cases, with a mean absolute error of 0.142.

HW1 on GitHub
Candlestick chart of Microsoft stock with 10-day and 30-day moving averages and trading volume
HW2: Microsoft prices with 10- and 30-day moving averages
Two t-SNE plots of node embeddings: scattered before training, cleanly separated clusters after
HW3: GAT node embeddings before and after training
Line chart: validation AUC and average precision rise from about 0.4 to about 0.87 over 400 training epochs
HW3: link-prediction validation AUC and AP
Spring 2025 · FinTech, NTU

Fraud detection, stock prediction and graph learning

HW1: credit-card fraud. On a highly imbalanced dataset, compared logistic regression, random forests, SVMs, a neural network and XGBoost, trained on the full data and with undersampling, to trade missed frauds against false alarms.

HW2: stock prediction. Microsoft price prediction with vanilla RNN, LSTM and GRU models from candlestick, moving-average and stochastic-oscillator features, plus a bonus FinBERT news-sentiment variant.

HW3: graph learning. GCN and GAT node classification on a product co-purchase network, then link prediction, which reached about 0.88 validation AUC.

The same course's final project won the Financial Technology Outstanding Innovation Award and became the GAP-TGN paper.
Pi SHIELD framework: the user takes an image, the Raspberry Pi blacks out the face, the cloud detects objects and returns a mask, and the Pi combines the original image with the mask
How a photo moves through Pi SHIELD: faces are blacked out on the Pi before anything reaches the cloud
Scatter plot of face detectors by inference time and average precision: OpenVINO is fast and accurate, MTCNN and YOLOv8 are accurate but slow
Speed vs. accuracy of the face detectors we compared
December 2024 · Mobile Pervasive Intelligence (CSIE5411), NTU

Pi SHIELD

A Raspberry Pi detects and anonymizes faces on the device, so no raw biometric data leaves it; a server then runs YOLOv8 on the anonymized frames and the Pi puts the results back on the original image. We benchmarked face detectors for the Pi, and OpenVINO gave the best accuracy for its latency.

Also in this course, with the same team: Bluetooth Low Energy beacon advertising and light-based communication between two Raspberry Pis, and Kubernetes service deployment with custom schedulers on a two-node Minikube cluster.

GitHub
The Plantita sensor board wired to a potted plant, with a hand holding one of its leaves
The sensor board in a plant pot
Plantita architecture in three layers: hardware sensors, a Flask server with Plant.id and Groq LLM APIs, and a LINE bot interface
System architecture: hardware, processing and LINE interface
Introduction to AIoT, TIGP

Plantita

A plant-care assistant: an Arduino Nano 33 BLE Sense Rev2 reads temperature, humidity, pressure and soil moisture; a Flask server uses the Plant.id API to identify the species and check plant health, and an LLM (via Groq) turns it all into care advice delivered through a LINE bot. Built with Zow Ormazabal.

GitHubDemo

Publications

Papers.

Google Scholar
  1. Review
    PRISM-CI: Bridging Low-Level Descriptors and Foundation Model Representations via Cross-Phase Ensemble Fusion for Cognitive Impairment Pre-Screening

    Screens for cognitive impairment from the voice and face of older adults during a word-recall test, with no transcript needed; held-out AUROC 0.761 on 276 participants.

    E. Sy et al. · IEEE Journal of Biomedical and Health Informatics, under review

  2. 2026
    Detecting Information Channels in Congressional Trading via Temporal Graph Learning

    A temporal graph network that links U.S. congressional trades to lobbying and campaign finance, plus Capitol Gains, a point-in-time dataset for 2013–2025.

    B. P. Roodman*, E. Sy*, J. X. Atero Vázquez, Y.-S. Huang, C. Lin, C.-J. Wang · ICLR 2026 Workshop on Advances in Financial AI · *equal contribution

  3. 2025
    Ensemble BERT Techniques for Financial Sentiment Analysis and Argument Understanding with Linguistic Features in Social Media Analytics

    Combines fine-tuned language models with classic classifiers for financial argument mining: 77.08% macro-F1 on argument units and 57.90% on relations.

    E. Sy, T.-C. Peng, H.-Y. Lin, S.-H. Huang, Y.-C. Chang, C.-P. Chung · Journal of Information Science and Engineering 41(3)

  4. 2024
    Enhancing Automated De-identification of Pathology Text Notes Using Pre-trained Language Models

    Fine-tuned T5 models plus rule-based post-processing to remove patient identifiers from pathology reports; macro-F1 up 14% and 30% over the baselines on the two tasks.

    Y.-Z. Huang, T.-C. Peng, H.-Y. Lin, E. Sy, Y.-C. Chang · IW-DMRN 2024, Springer CCIS vol. 2148

  5. 2023
    TMUNLP at the NTCIR-17 FinArg-1 Task

    Four BERT-family models voting together to find arguments in earnings calls and social media; 1st in argument unit identification and 3rd in relation detection.

    H.-Y. Lin, E. Sy, T.-C. Peng, S.-H. Huang, Y.-C. Chang · NTCIR-17, Tokyo · Best Oral Presentation

  6. 2023
    TMUNLP at the NTCIR-17 MedNLP-SC Task

    Detecting adverse drug events in social-media posts, using data augmentation to cope with rare event types.

    Y.-Z. Huang, E. Sy, Y.-X. Lin, Y.-L. Hsieh, Y.-C. Chang · NTCIR-17, Tokyo

  7. 2023
    Fine-Grained Argument Understanding with BERT Ensemble Techniques: A Deep Dive into Financial Sentiment Analysis

    The first version of our BERT-ensemble approach to fine-grained argument mining in financial narratives.

    E. Sy, T.-C. Peng, S.-H. Huang, H.-Y. Lin, Y.-C. Chang · ROCLING 2023, Taipei · Poster

Experience

Work and teaching.

Download CV
2026

AI/ML Engineering Intern

AUO Mobility Solutions Corporation (AMSC), Hsinchu

LLM tool-calling backend for an edge hardware product. Completed the NTU International Mentorship Program with an A+ (verified certificate).

National Taiwan University seal 2025 – now

Teaching Assistant, Frontiers of AIoT

NTU TIGP-AIoT

Course administration, classroom logistics and guest-lecture coordination.

Academia Sinica seal 2025 – now

Researcher, BioASP Lab

Research Center for Information Technology Innovation (CITI), Academia Sinica

Ph.D. research with Prof. Yu Tsao on multimodal pre-screening for cognitive impairment (PRISM-CI).

Taipei Medical University seal 2023 – 2024

Part-time Data Scientist

Bilingual Education Promotion Center, TMU

Statistical models and survey analysis behind the university's English-medium instruction strategy.

Taipei Medical University seal 2023 – 2024

Teaching Assistant, Python Programming

Taipei Medical University

Led lab sessions, helped students with Python syntax, debugging and assignments, and graded Jupyter notebooks with code-level feedback.

2021 – 2022

Mandarin Bilingual Finance Process & Ops Analyst

Accenture

Led the APAC invoice review team for Australia, China, Hong Kong and Taiwan: allocated volumes across the team, kept a 0% error rate, and led weekly client meetings for 7 months.

Toolkit

What I work with.

Modelling

  • PyTorch
  • TensorFlow / Keras
  • Hugging Face Transformers
  • Graph neural networks (DGL)
  • Temporal graph networks
  • scikit-learn
  • ONNX model export

Used in PRISM-CI, GAP-TGN and argument mining

Speech and vision

  • Whisper
  • AV-HuBERT
  • Acoustic features (MFCC)
  • Facial action units
  • YOLOv8
  • OpenVINO

Used in PRISM-CI and Pi SHIELD

LLM systems

  • Tool calling
  • Retrieval-augmented generation
  • Agentic workflows
  • On-device inference and LLM serving
  • KV-cache reuse, speculative decoding, flash attention
  • LLM evaluation and model bake-offs
  • Speech-to-text and text-to-speech pipelines

Used in the AUO internship

Research methods

  • Ablation studies
  • Benchmarking against published baselines
  • Held-out and leave-one-subject-out validation
  • Significance testing, including pre-registered tests
  • SHAP, mixed-effects models, Mann-Whitney U
  • Reproducible code releases
  • Literature review

Used in PRISM-CI, GAP-TGN and the TMU data science work

Engineering and infrastructure

  • Python, C
  • Linux and Docker deployment
  • Networking: VPN, DNS filtering, subnets
  • A self-hosted home lab
  • AI coding agents (Claude Code)
  • Git, LaTeX, Streamlit
  • Raspberry Pi and Arduino

Used in Pi SHIELD, Plantita, a home lab and this website

Languages

  • English (fluent)
  • Filipino (native)
  • Mandarin (HSK 4)

Used in teaching, research and work in Taiwan and the Philippines

Education

Degrees.

National Taiwan University sealAcademia Sinica seal
2024 – now

Ph.D., Artificial Intelligence of Things

National Taiwan University and Academia Sinica, Taiwan International Graduate Program (TIGP-AIoT)

Running GPA 4.25 / 4.3 · TIGP Scholarship

Taipei Medical University seal
2022 – 2024

Master's, Data Science

Taipei Medical University, Taipei

GPA 4.0 / 4.0 · Class Valedictorian

FEU Institute of Technology seal
2015 – 2021

Bachelor's, Electronics Engineering

FEU Institute of Technology, Manila

GPA 3.2 / 4.0

Recognition

Awards and certifications.

Talks, activities and engagements.

Eugene giving two thumbs-up under the AUO logo in the company lobby
October 2026

NTU International Mentorship Program, Certificate of Completion (A+)

Completed National Taiwan University's 2026 International Mentorship Program with a two-month internship at AUO (June 29 – August 28, 2026), graded A+ “with outstanding performance”. The internship work was an LLM tool-calling backend for an edge hardware product.

Verify the credential
Eugene and the other award winners holding their certificates in an NTU classroom June 2025

Financial Technology Outstanding Innovation Award

Awarded to my team (Group 5) in National Taiwan University's 113-2 FinTech Final Project Contest. I kept working on it afterwards as a collaborator with Prof. Chuan-Ju Wang's CfDA Lab, and it grew into the GAP-TGN paper at the ICLR 2026 Workshop on Advances in Financial AI.

Prof. Che Lin's post about the course
Eugene with the third-place team and the presenter holding the prize board at the 2025 Taiwan in My Eyes 120h ceremony March 2025

Third Place, the 11th “Taiwan in My Eyes 120h – Power Up You&Me” (2025)

A five-day sustainability program run by the CTCI Education Foundation, visiting a waste-to-energy plant, a wind farm, a solar-powered shrimp farm, an offshore solar installation and the Future Earth Taipei symposium at Academia Sinica. On the AIoT team, I linked each site to the UN Sustainable Development Goals and proposed AIoT extensions such as smart energy monitoring and predictive maintenance.
September 2024 – now

Taiwan International Graduate Program (TIGP) Scholarship

Academia Sinica's scholarship for students of the Taiwan International Graduate Program. It funds my Ph.D. in Artificial Intelligence of Things, run jointly with National Taiwan University.
2024 – now

TIGP-AIoT Program Scholarship

Awarded each year of my Ph.D. by National Taiwan University's TIGP-AIoT program, in its Outstanding Research Performance and Personal Research Support categories.
Eugene receiving the Fubon Life thesis award certificate and trophy on stage September 2024

2024 Fubon Life Management Doctor and Master Thesis Award

Outstanding Paper Award (佳作) in the Master's category of Information Management and Innovative Application, for my master's thesis, “Ensemble BERT Approach for Argument-Driven Sentiment Analysis in Financial Text Mining Applications”. The award is run by the Chinese Management Association and Fubon Life Insurance.
Eugene and teammates taking a selfie with their Ministry of Education AI CUP certificates at the award ceremony March 2024

Ministry of Education 2023 AI CUP Award, Honorable Mention

Honorable Mention (佳作) in the 2023 AI CUP Competition on Privacy Protection and Medical Data Standardization, “Decoding clinical cases and letting data tell stories”, Taiwan's national AI competition for university students. The task was de-identifying pathology reports and standardizing medical data.
Eugene and Zhi-Chi Ye at the podium at the 2024 Taipei Medical University commencement, next to a university speaker in a gold-trimmed gown June 2024

Class Valedictorian

Class Valedictorian of the Master's in Data Science at Taipei Medical University, graduating with a 4.0 / 4.0 GPA. At the May 27, 2024 commencement, I spoke on behalf of the graduates together with Zhi-Chi Ye of the School of Medicine.

TMU's story on the 2024 graduation
The TMUNLP team giving a thumbs-up in front of the FinArg-1 poster at NTCIR-17 in Tokyo December 2023

NTCIR-17 Best Oral Presentation Award

For our talk on “TMUNLP at the NTCIR-17 FinArg-1 Task” at the 17th NTCIR Conference, National Institute of Informatics, Tokyo (December 12–15, 2023). Shared with my co-authors. Among nearly 20 teams, ours placed 1st in Argument Unit Identification and 3rd in Argument Relation Detection and Classification.

TMU's news story on the team
Lab members at the booth demonstrating live speech enhancement to a crowd of visiting students, beside the “AI Sees What You Hear” poster October 17, 2026

Live demo at the Academia Sinica Open House

At Academia Sinica's 2026 Open House, I introduced visiting students to our lab's audio-visual speech enhancement (AVSE) system and ran it live at the CITI booth. When several people talk at once, AVSE watches the speaker's face as well as listening, and uses both to bring that one voice out of the noise.
Eugene presenting the outline slide of the ChatGPT Plus Me workshop February 2024

“ChatGPT Plus Me”: Leveraging LLMs for Teaching Assistants

I designed and led an interactive workshop at Taipei Medical University for bachelor's, master's and Ph.D. teaching assistants: how ChatGPT and other large language models work, their strengths and limits, prompt engineering, and practical uses such as announcements and emails, assignment checking, and building slides.
January 2023

Innovative Proposal Excellence Award

Given for a research proposal at the AI @ TMU Workshop 2023, Taipei Medical University.
Eugene giving a thumbs-up next to his ROCLING 2023 poster
October 2023

Poster presentation at ROCLING 2023

Presented my first-author paper, “Fine-Grained Argument Understanding with BERT Ensemble Techniques: A Deep Dive into Financial Sentiment Analysis”, as a poster at ROCLING 2023, the 35th Conference on Computational Linguistics and Speech Processing in Taipei.

Read the paper
The field-trip group with community hosts at the Penglai workshop, its walls hung with Saisiyat vine-woven crafts
August 12–13, 2023

Miaoli Field Trip: Indigenous ecology and sustainable development

A two-day Taiwan–Philippines exchange run by the Center for Taiwan-Philippines Indigenous Knowledge, Local Knowledge, and Sustainable Studies (CTPILS) at National Chengchi University, with Filipino government and NGO representatives and students in Taiwan.

We visited the Saisiyat community of ray'in (Penglai) in Nanzhuang, Miaoli, to see how its forestry cooperative runs an under-forest economy without cutting trees: shade-grown mushrooms, bee farming and free-range chickens, plus old forest trails and traditional vine weaving.

CTPILS trip report
Eugene and two fellow student volunteers in SIGIR 2023 shirts, taking a selfie at a work table with laptops July 2023

Student Volunteer, SIGIR 2023

Volunteered at the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2023), July 23–27, 2023, at the Taipei International Convention Center.
The OGE Squad team of international and local students posing together on the TMU campus October 2022 – December 2023

Head of the Events Team, OGE Squad

Led the events team of Taipei Medical University's Office of Global Engagement student squad, organizing events that bring international and local Taiwanese students together: the TMU Christmas Party, a cooking competition, the International Graduates Farewell Party and International Cultural Dance Night. I ran each event from idea to evaluation with a mixed team of local and international students.
Eugene helping a group of high-school students work through logic-circuit worksheets at a workshop table March 2018

Instructor, Introduction to Logic Circuits Workshop

Part of a Basic Electronics class for senior high school students at FEU Institute of Technology, run with student volunteers: logic gates and circuit design, ending with each student building a working circuit on a breadboard with switches and ICs.

Get in touch.

I'm open to research collaborations and internships in clinical AI, financial machine learning and on-device language systems.

d13949006@ntu.edu.tw

LinkedInGitHubGoogle Scholar

LINE: @syeugene