Efficient deep learning. Software for learning.

Uranik Berisha

PhD researcher & software developer.

I’m a PhD researcher at Bosch and the University of Lübeck. I study how to make neural networks smaller and faster, and co-founded Fletza to build tools that help people learn.

Explore my timeline LinkedIn Email Google Scholar

A few paths, in parallel.

Experience & education

EducationWork

Jan 2024 — present PhD in Machine Learning University of Lübeck An industrial doctorate on making trained models more efficient.

I’m pursuing an industrial doctorate with Bosch and the University of Lübeck. My research asks which parts of a trained neural network matter most, and how to reduce its computation while preserving useful behaviour.

I focus on efficient deep learning and computer vision, particularly pruning and extracting experts from Vision Transformers. My doctoral thesis, Optimization and efficiency of neural networks, is in progress.

Jan 2025 — present Fletza Co-founder & Full-Stack Developer An AI-powered learning platform.

I co-founded Fletza and developed the platform end to end, building both the frontend and backend. It turns lecture materials into flashcards, quizzes and exam questions, with spaced repetition and automatic feedback.

Jan 2024 — present Robert Bosch GmbH PhD Student · Stuttgart From model-compression ideas to evaluated methods.

At Bosch, I develop and evaluate methods for making vision models smaller and faster. My work combines activation analysis, structured pruning and Mixture-of-Experts extraction from existing networks.

This research has led to papers at CVPR and ICCV, an ICCV oral presentation, and two pending patent applications.

Oct 2021 — Jun 2023 M.Sc. Computer Science TU Darmstadt Data science, probabilistic models and learning over time.

I specialised in data science at TU Darmstadt, studying machine learning, probabilistic modelling and statistical analysis. I completed the degree in three semesters with a final grade of 1.2.

Deutschlandstipendium recipient.

My master’s thesis, Progressive Probabilistic Circuits, explores Sum-Product Networks with task-specific models and lateral connections inspired by progressive neural networks.

The construction preserves tractable inference. Experiments reveal unstable training and restricted knowledge transfer: in its tested form, the approach is not yet practical for continual learning.

Examiner: Dr. Martin Mundt · Supervisor: Steven Braun.

Jan 2020 — Dec 2025 JLU Giessen Systems Administration Examination systems, statistics and Python / SQL automation.

Alongside my studies and later my doctorate, I worked on examination administration at JLU Giessen. I maintained the system, prepared student statistics and supported the recurring work around course and examination data.

I built a Python and SQL tool to automate course imports, replacing a manual task with a repeatable workflow.

Apr 2019 — Sep 2021 B.Sc. Computer Science THM Giessen Software engineering, web development and an interactive learning tool.

At THM, I studied software engineering and system design while also pursuing mathematics at JLU. Web development became a way to turn formal ideas into tools people can explore.

With Thomas Moissl, I developed Parser Generator, an interactive learning environment for compiler construction. My bachelor’s thesis extended the tool with LR(1) parsing and examined its implementation and learning support.

Final grade: 1.3.

Jul — Aug 2019 Robert Bosch GmbH Working Student · Lollar

I spent the summer of 2019 at Bosch in Lollar, supporting the assembly and production of heating boilers. It gave me first-hand experience of industrial production before I later returned to Bosch for machine-learning research.

Oct 2018 — Sep 2021 B.Sc. Mathematics JLU Giessen Geometry, algebra and the mathematical structure behind software.

I studied geometry and algebra at JLU while completing computer science at THM. Working with definitions, proofs and abstract structures developed the kind of formal reasoning I now bring to algorithms and models.

My mathematics thesis examined the LR(1) method for recognizing deterministic context-free languages, connecting formal language theory with the Parser Generator project.

Final grade: 1.5.

2018 · Completed in parallel Abitur IB Diploma Herderschule Giessen Two school qualifications pursued in parallel.

At Herderschule Giessen, I completed the German Abitur and the International Baccalaureate in parallel. The IB qualification was awarded as a bilingual diploma.

I received the Abitur in May 2018 and the IB diploma in July 2018.

What can a network do without?

Research & patent applications

Google Scholar

ICCV 2025Oral presentation0.57% of submissions Compress a trained network while preserving what matters.Variance-Based Pruning for Accelerating and Compressing Trained Networks

01 Activation statistics

Estimate each neuron’s mean and variance.
02 Variance pruning

Replace low-variance activations with their means.
03 Mean-shift compensation

Fold the means into the output bias.

Measure hidden activations in a trained MLP. Remove neurons that vary least, then preserve their mean contribution in the output bias. Both linear layers become smaller; input and output dimensions stay unchanged.

Official PDF ↗Code ↗arXiv ↗

Why variance, rather than magnitude?

A near-constant activation can be large yet predictable. Its expected squared mean-replacement error equals its variance.

Why keep the mean?

Removing an activation loses its average contribution. Folding that constant into the bias preserves it without computing the neuron.

b_{2}^{'} = b_{2} + W_{2, R} μ_{R}

R: removed neurons.

What changes at inference?

Entire hidden neurons disappear from both weight matrices. Bias compensation is precomputed, with no extra inference step.

Uranik Berisha, Jens Mehnert & Alexandru Paul Condurache.

CVPR 2025Conference paper Find the specialists already inside a pretrained model.Efficient Data Driven Mixture-of-Expert Extraction from Trained Networks

01 Cluster

Group hidden activations with HDBSCAN.
02 Extract

High-variance neurons form overlapping experts.
03 Route

Route each token by input-mean similarity.

Starting with a trained Vision Transformer, we discover activation clusters, extract their neuron subsets, and route each new token to one expert. Learned weights are reused; per-token computation is reduced.

Official PDF ↗arXiv ↗

How many experts are there?

HDBSCAN discovers the cluster count from activation data, rather than requiring a fixed expert count.

Can experts share neurons?

Yes. Experts select high-variance components independently. Shared weights are stored once; neurons unused by every expert can be removed.

How is a token routed?

Compare it with each expert’s mean input using cosine similarity, then evaluate only the best-matching expert.

Uranik Berisha, Jens Mehnert & Alexandru Paul Condurache.

Patent applications

Application pending

Network pruning & compression

Related ICCV publication ↗

Application pending

Mixture-of-Experts extraction

Related CVPR publication ↗

Ideas, put to work.

Software projects

The problem, the implementation, and my part in it.

2025 — presentCo-founder & developer Turn course material into something you can practise.Fletza

Overview

Fletza turns course material into flashcards, quizzes and exam questions. Spaced repetition brings topics back for review, and automatic feedback helps students check their understanding.

My role: Co-founder and full-stack developer.
Contributions: Developed the frontend and backend, including AI-generated study sets, user authentication and subscription payments.
Built with: React & Material UI; TypeScript/Deno Edge Functions; Supabase (PostgreSQL); OpenAI API & Stripe.

01 / Start withCourse material
Lecture notes and explanations.
02 / Turn it intoActive practice
Flashcards, quizzes and exam questions.
03 / Keep learningReview & feedback
Spaced repetition and automatic feedback.

Fletza ↗Fletza in my timeline ↑

2021Two bachelor’s theses Learn compiler construction by building the parser yourself.Parser Generator

Overview

A web-based learning environment for context-free grammars. Students work out FIRST and FOLLOW sets, construct states and complete parser tables, with corrections and optional solutions along the way.

My role: Co-developer with Thomas Moissl.
Contributions: Extended the learning tool with LR(1) parsing, connecting formal language theory with interactive exercises.
Built with: JavaScript, HTML & CSS.

Grammar → states → parser table.

Students construct the parser themselves. Corrections make errors visible at the step where they occur.

Parser Generator ↗

Live project ↗Source code ↗

TU Darmstadt · 2022Serious Games project Make focused work visible through a growing virtual garden.Plantivity

Overview

Each project has a plant. Timed work sessions sustain its growth; neglected projects wither. Time statistics provide the practical record, while coins, items and achievements add a reason to return.

My role: Co-developer with Thomas Moissl for the Serious Games course at TU Darmstadt.
Contributions: Co-developed an iOS app linking focused work sessions to a virtual garden, with project tracking, plant rewards and progress statistics.
Built with: Swift, SwiftUI & Core Data.

A timer with a longer-term reward.

Projects and sessions provide the time record. Plants, coins and achievements turn that record into visible progress.

Project trailer · 28 seconds

Project report · PDF ↗Trailer · MP4 ↗

2021 · Open sourceContributor More expressive event subscriptions for industrial systems.open62541 · EventFilter

Overview

An extended OPC UA EventFilter lets subscriptions select event fields and combine conditions to decide which notifications reach a client.

My role: Open-source contributor.
Contributions: Implemented initial SelectClause and WhereClause validation and recursive filter evaluation, with tests and examples.
Built with: C & CMake.

Conditions and selected event fields

Filter evaluation checks an event against combined conditions, then selects the fields to include in its notification.

Event conditions

Severity ≥ 500
AND Severity < 900

Notification fieldsMessage, Severity

Project & source code ↗

Beyond the paper.

Talks & teaching

Presentations, workshops, and shared understanding.

ICCV 2025 · Oral presentation

Variance-Based Pruning

Presented the motivation, method and practical relevance of our research to more than 1,000 attendees.

64 orals / 11,239 submissions · 0.57%

University of Stuttgart · Workshop

Generative AI, Machine Learning & Deep Learning

From next-token prediction and GPT architecture to data collection, pretraining, instruction tuning and learning from feedback.

BVDW Academy · Professional training

AI assistants & agents at work

I co-develop and teach practical courses on working with AI assistants and agents: choosing tools, structuring tasks, checking results and handling work data responsibly.

ifb · Independent lecturer

Seminars for employee representatives

I work as an independent lecturer with ifb, preparing seminar materials and practical sessions for employee representatives. My teaching focuses on clear explanations and active participation.

Research, code & communication.

Skills & languages

Programming languages

PythonAdvanced

2020 · ChefBot
Rule-based recipe chatbot built with Python and AIML.
2020–2025 · JLU Giessen
Python and SQL automation for course imports and examination data.
2021–2023 · TU Darmstadt
Python-based data science and the Progressive Probabilistic Circuits thesis.
2024–present · Bosch & University of Lübeck
Research on efficient deep learning and model compression.

JavaScript / TypeScriptProficient

2021 · Parser Generator
JavaScript implementation of the interactive learning tool, developed with Thomas Moissl.
2025–present · Fletza
React frontend and TypeScript edge functions for study-material generation and payments.

SQLAdvanced

2020–2025 · JLU Giessen
Automated course imports and worked with examination data and student statistics.

C / C++Proficient

2021 · open62541
C contributions to an industrial OPC UA stack.

JavaWorking

2019–2021 · THM Giessen
Learned Java during my computer science studies at THM.

SwiftProficient

2022 · Plantivity, TU Darmstadt
Learned and applied Swift and SwiftUI while co-developing an iOS productivity app.

Tools & technologies

PyTorchAdvanced

2023 · Master’s thesis
Parameter optimization and probabilistic-circuit experiments in PyTorch.
2024–present · Bosch & University of Lübeck
Vision Transformer pruning and expert-extraction research.

NumPy, Pandas, SciPy & scikit-learnProficient