User-centric data lake architectures for cultural and tourism informatics

Abstract: The progress in Cultural and Tourism Informatics has substantially influenced the way people perceive, analyze, communicate and understand culture and cultural tourism. Emerging data sources, including social media, digitized cultural material, and Internet of Things devices, have enabled society to enhance and personalize the cultural experience. However, they have also generated a vast amount of new content, which must be stored and effectively managed to provide valuable assets. While data management is crucial for the advancement of both the Cultural Heritage and Cultural Tourism sectors, current solutions are fragmented, physically dispersed, requiring specialized IT expertise for their implementation, and considerable IT experience to operate, even for basic tasks. At the same time, as cultural and touristic information becomes increasingly accessible through a multitude of heterogeneous sources, such as social networks, official repositories, community platforms and open data initiatives, capturing all the available content can result in highly noisy datasets. For example, although well-known landmarks generally appear across these sources, lesser-known attractions are also recorded, yet with different levels of detail, leading to content that may be incomplete, overlapping, or complementary. To facilitate comprehensive access to this extensive information, it is essential to implant harvesting and consolidation methodologies that can gather, reconcile, and unify distributed content related to the same entities. In this thesis we study the problem of both human-assisted and automated data collection, through digital questionnaire-like and web scraping procedures respectively, in the fields of cultural tourism and transportation. We thoroughly explore various data storage, management and analysis architectures, we examine different levels of user collaboration to increase their engagement in cultural tourism research, while paying special attention to user access control for content security reasons and to user-friendly interfaces to target users with limited or no IT knowledge. In addition to the above, we focus on data resolution approaches to harmonize duplicate records referring to the same entity and consolidate complementary ones, while we utilize Machine Learning models to extract high-value insights from the captured content. Initially, we present Hydria, an online data lake designed for users who lack an IT background, enabling them to harvest, store, organize, analyze, and distribute heterogeneous and complex Cultural Heritage and Tourism data. Hydria offers a zero-administration, zero-cost, integrated framework that allows researchers, museum curators, and other stakeholders in the cultural tourism sector to effortlessly: (i) implement data acquisition services (such as social media scrapers, focused web crawlers, dataset imports, and questionnaire forms), (ii) create and manage adaptable customizable data repositories leveraging traditional databases, (iii) share entire datasets or specific horizontal/vertical data segments with other stakeholders via appropriate publish/subscribe mechanisms, (iv) search, filter, and analyze recorded content using an intuitive yet powerful graphical query engine and visualization tools, and (v) conduct user management and access control tasks on the stored data. Subsequently, we propose EnQuest, an expansion of Hydria equipped with a schema-less, document-oriented NoSQL database. In this architectural design, massive datasets can be ingested without affecting the operation of the data lake. In addition to the features provided by Hydria, EnQuest was developed in a way that allows access to unauthorized users (i.e., those without data lake credentials) for data acquisition and analysis tasks, while its data visualization service was further extended to enable its users to create more sophisticated visualizations by combining multiple data fields simultaneously. The proposed system is entirely open-source, designed upon big data tools and principles for data storage, analytics production, and knowledge sharing, targeting both performance and usability. The results of extensive performance evaluations compared to its predecessor (i.e., Hydria) highlighted EnQuest’s robustness and stability by at least an order of magnitude across all operations, as the workload on both data lake alternatives steadily increased. Finally, we put forward a machine learning-driven framework for harvesting, homogenizing, and augmenting cultural and tourism data across multilingual, diverse sources and apply it to both data lake approaches. Our approach addresses entity resolution, duplicate detection, and complementary content harmonization by combining both pre-recorded geographic information and various toponym string similarity metrics, establishing a basis for an enriched and unified representation of attractions and points of interest. Furthermore, the captured content is enhanced by leveraging Natural Language Processing methods, including automated text translation libraries, named entity recognition language models, and sentiment analysis pipelines, to yield actionable insights, such as cultural tourism trajectories. The framework is designed to facilitate scalable integration pipelines and is suitable for deployment in applications focused on tourism promotion, digital heritage, smart travel services, and transportation. It is worth noting that all aforementioned systems were implemented, deployed and used in real-life scenarios including European and national research projects bringing user feedback and tangible results.

Service- centric UAV edge systems: design and field validation

Abstract: Unmanned Aerial Vehicles (UAVs) have experienced rapid adoption across multipledomains; however, most existing systems remain monolithic, mission-specificand tightly coupled to the aircraft itself. Traditional deployments typically relyon isolated platforms, manual piloting and application-specific configurations,limiting scalability and interoperability. As the demand for complex, large-scaleand data-driven services increases, there is a clear shift toward integrating UAVsinto broader distributed ecosystems where they operate as service providersrather than standalone assets. This transition introduces significant challenges,including reliable connectivity in remote or infrastructure-poor environments, efficient management of distributed UAV fleets and applications and the needfor seamless orchestration across heterogeneous network and computing layers. Addressing these challenges requires a paradigm shift from UAV-centric designto service-centric, ecosystem-driven architectures. This thesis addresses this gap by proposing and engineering a service-centric UAV edge system, transforming conventional UAVs into autonomous, network aware and computationally capable edge nodes. The main objective is to enableUAVs to execute application-level services, while being abstracted from low-levelflight operations. To achieve this, the work integrates hardware and software components, including onboard computing platforms, adaptive communication subsystems (4G/5G, NB-IoT and satellite links) and lightweight container orchestration mechanisms. Particular emphasis is placed on hardware integration, including power management, communication modules and autopilot–companion computer interaction. The thesis further introduces a unified software stack enabling containerized service execution, mission-aware control and network adaptives cheduling. The proposed system evolves UAVs into components of adistributed ecosystem capable of supporting autonomous service deployment and execution.The proposed architecture is validated through extensive laboratory and real-world field trials, focusing on both system performance and operational feasibility. Experimental evaluations include autonomous inspection missions, edge processing services and communication performance analysis under differentnet work conditions. Special emphasis is given to the use of terrestrial and satellite Narrowband IoT (NB-IoT) as a novel alternative for UAV communications in remote areas, enabling store-and-forward data transmission and energy efficient connectivity. Additionally, advanced networking concepts such as aerial relay nodes and hybrid communication architectures are experimentally assessed, demonstrating the feasibility of maintaining connectivity in underserved environments. These trials provide quantitative insights into power consumption, latency, throughput and system stability, validating the practical applicabilityof the proposed approach. The thesis is organized as follows. Chapter 1 introduces the motivation andcontributions. Chapter 2 reviews background and related work. Chapter 3 presents the UAV edge node system and architecture. Chapter 4 details theconnectivity layer evaluation. Chapter 5 describes the cloud–fog–edge ecosystem.Chapter 6 presents the field trial validation. Chapter 7 concludes with limitationsand future research directions.

Computer vision based rehabilitation and motor function assessment with clinical study and validation for upper limb recovery in post stroke patients

Abstract: Stroke-induced upper limb motor deficits remain one of the most prevalent and debilitating outcomes among survivors, significantly impairing daily functioning and quality of life. Traditional rehabilitation practices, while clinically effective, are often constrained by their dependence on in-person evaluations, the subjectivity of observational assessments, and limited accessibility for individuals facing mobility or logistical barriers. These challenges underscore the growing necessity for remote, objective, and scalable rehabilitation and evaluation tools that can support both patients and clinicians. In response to this pressing need, this Thesis explores the integration of computer vision technologies into post-stroke upper limb rehabilitation frameworks, aiming to deliver accurate, real-time motor performance assessments without the requirement for specialized sensors or equipment. The work presented herein introduces both desktop-based and mobile-oriented systems that leverage standard cameras and modern computer vision techniques to simulate established rehabilitation protocols, such as the Box and Block Test (BBT) and the Sollerman Hand Function Test (SHFT). Through algorithmic tracking of hand movements and gesture recognition, the proposed applications automatically compute clinically relevant metrics in a non-invasive, user-friendly manner. The Thesis outlines the complete software architecture of these systems, including interaction modeling, gesture interpretation, and augmented feedback mechanisms. In addition to providing an immersive and engaging rehabilitation experience, both implementations were rigorously performance-evaluated with respect to latency, robustness, and system responsiveness, under a variety of computational and environmental conditions. These evaluations confirm the technical feasibility of delivering high-fidelity rehabilitation tools across both platforms, enhancing the accessibility and practicality of home-based motor recovery interventions. To support clinical relevance and interpretability of the measured outcomes, a comprehensive baseline score normalization study was conducted using a cohort of healthy individuals to establish normative performance ranges. This process enables direct comparison between users and facilitates the detection of motor impairments with greater precision. Furthermore, a clinical validation study was carried out involving older post-stroke individuals, wherein the system’s outputs were compared against conventional therapist-administered evaluations. Results indicate strong concordance between automated and manual assessments, demonstrating the potential of computer vision-driven tools to augment clinical decision-making and empower patients to engage in effective, self-directed rehabilitation. Overall, this Thesis offers a robust, scalable, and clinically grounded framework for enhancing upper limb rehabilitation through accessible computer vision methodologies. The Thesis is organized into Sections. Following the Introduction (Section 1), Section 2 presents the computer vision foundations underpinning the system, including key algorithms, architectural choices, and real-time implementation strategies. It also examines augmented reality as an interactive medium for motor rehabilitation, discussing relevant design principles, healthcare use cases, and technical constraints. Furthermore, the section situates the work within established clinical rehabilitation standards by detailing upper limb rehabilitation protocols and motor assessments such as the BBT and the SHFT. Lastly, it reviews existing digital adaptations of these assessments, identifying limitations in current approaches and outlining integration challenges that this Thesis seeks to address. Section 3 describes the desktop-based system architecture, including virtual test modules and a custom exergame to enhance engagement. Section 4 extends the system to mobile platforms, highlighting design adaptations and deployment strategies for at-home use. Section 5 provides a comparative evaluation of technical performance metrics across platforms under varied conditions. Section 6 presents clinical validation via score normalization and trials with post-stroke individuals, assessing diagnostic and therapeutic outcomes. Finally, Section 7 summarizes the contributions and proposes directions for future research and system expansion.

Scalable data-driven enrichment analysis of short RNAs

Abstract: microRNAs (or miRNAs) are short RNA molecules that play a crucial role in the regulation of gene expression, i.e. the production of proteins, which are important structural and functional components of a cell. miRNAs silence genes by binding to them, thus blocking the production of the respective protein. However, the mechanisms underlying miRNA function are often very complex; combined with the fact that the number of miRNAs in a given organism ranges in the order of thousands, it becomes evident that utilizing experiments in a wet lab is a lengthy, arduous and often expensive process. Additionally, scientists may be interested to compare differences in miRNA expression between diseased and healthy individuals and need to quantify them, using statistical methods. For this reason, Bioinformatics researchers developed in-silico methods and algorithms like miRNA enrichment analysis, which is a statistical technique to predict whether a set of miRNAs is likely to affect a certain biological function. One of the most recent such approaches is the unbiased miRNA functional enrichment analysis, which relies on a considerably large number of set operations and consequently, it leads to execution times in the order of hours or days.In this thesis, we strive to make the unbiased miRNA enrichment, a computationally-intensive, data-driven analysis, more scalable by utilizing data management and other computer science techniques. Initially, we examine the performance of data structures called bit vectors in comparison to hash tables as a set representation technique and propose a new hybrid approach to reduce the execution time of the analysis. Moreover, we optimize miRNA enrichment by introducing two novel indices, utilized in order to reduce redundant set participation operations and filter out potentially insignificant associations between miRNAs and biological functions. Additionally, we showcase that the state-of-the art unbiased enrichment suffers from reduced sensitivity to false negatives and we propose a modification to its statistics engine in order to increase its quality. Furthermore, we propose an approach using supervised machine learning methods to predict approximate p-values in real time, instead of executing a full analysis. Finally, we introduce data management and processing techniques during the design of online miRNA analysis tools to achieve analyses in almost real-time; also, we try to address the need for a platform facilitating reproducible and scalable execution of containerized software in a Cloud environment, consisting of heterogeneous machines.

Single-service or multi-service loss models for cloud radio access networks

Abstract: The cloud radio access network (C-RAN) is considered as a promising fifth generation (5G) network architecture. The C-RAN architecture implies that a number of distributed base stations (BSs) form a centralized network. Each BS is split into two components: the remote radio head (RRH), that includes the antennas and the baseband signal processing server, named baseband unit (BBU). The BBUs are centralized and implemented as virtualized BBUs (V-BBUs) by a computer data center. The pool of V-BBUs is connected to the RRHs via fronthaul links. We consider the C-RAN from the aspect of a loss system. In order for an arriving call to be served by the C-RAN, resource units should be allocated by the RRH (in which the call arrived) and the V-BBU pool (i.e., by the data center). The capacity of the RRH is expressed in radio resource units, while the capacity of the V-BBU pool is expressed in computational resource units. The RRHs are distinguished into homogeneous, when they have same capacity, and into heterogeneous, when they have different capacity of radio resource units. When all calls require the same amount of resource units, then we have single-service traffic, otherwise, when different calls require different amount of resource units, then we have multi-service traffic. In this dissertation, C-RAN systems are examined as loss models. In the first part, we present 14 loss models (of which 12 are novel and proposed), that consider the following cases: the C-RAN has either homogeneous or heterogeneous RRH, the call traffic is either single-service or multi-service and the call arrival process is either random (i.e., Poisson process), quasi-random (i.e., calls are generated by a finite number of users), or compound Poisson (i.e., calls arrive in batches, the size of a batch is random, while the batch arrival follows a Poisson process).The second part of the dissertation examines multi-service loss systems under the so called Threshold Call Admission (TCA) policy. Two loss models are presented (of which one is novel and proposed) that consider a random or quasi-random call arrival process. In the multi-service loss models with TCA policy, calls of the same service-class are distinguished into: new calls and handover calls. The TCA policy sets a threshold (that express a number of in-service calls) for each service class. When the threshold is reached, then new arriving calls are blocked and lost, while handover calls are accepted, in case there are available resource units for the service-class to which they belong. All models, in this dissertation, are analyzed as continuous time Markov chains and for each such Markov chain it is proved that there exists steady-state distribution which is expressed by a product form solution. Based on the product form solution, a convolution algorithm is proposed for the calculation of the congestion probabilities and the system’s capacity utilization. The analytical results of the convolution algorithms were validated with the simulation results via examples.

Abstract: In this thesis the successful testing of an electronic nose system is presented, as well as all the required research to achieve that goal. This electronic nose is intended to detect the presence of trapped humans in confined spaces and, also, air-related hazardous conditions in collapsed buildings during search and rescue operations in an urban environment. Initially, the field of interest for which this work provides a solution is presented. Existing methods, with and without gas sensors, are presented and the environment in which the electronic nose is intended to work in is investigated from a measurement and detection perspective. Various sensing technologies and their capabilities are presented. Also, the anticipated target gases/compounds which are indicators of human presence are listed and an estimation regarding the feasibility of detecting them is produced, given some specific constraints that must be met in order for the end result to be practically usable in real conditions. The research and resulting developments were performed incrementally by first confirming, theoretically and then experimentally, the feasibility of the proposed approach – which is to use sensors for the measurement of gas concentrations, temperature, humidity and air pressure. After the successful completion of the feasibility test of the proposed approach, an electronic nose system capable of detecting human presence in confined spaces was designed and successfully tested under near realistic conditions. The testing method of this system confirmed the ability of the system to detect human presence by performing a search on the surface of the building debris. Given this success, a new electronic nose system, which was designed and tested under realistic conditions, was installed on a robotic platform capable of moving inside collapsed building structures. This system can detect human presence and many typical air related hazards inside building rubble. While the functionality of the system was increased and its volume reduced, the operator interface was drastically improved with the introduction and implementation of logic that produces simple outputs/indicators regarding the detected conditions. Finally, alternative uses of the system are investigated, as well as the required adjustments in each case.

Abstract: In the recent years, the collection of more and more data instances, has increasingly led to an abundance. After the investigation of efficient ways to deal with storing, managing and collecting of large-scale or diverse data, the research interest of the scientific community has shifted into the extraction of meaningful information from such collections. Deep learning lends itself particularly well to the process of extracting valuable information. Deep learning methods thrive with large-scale datasets. Due to their ability to learn alternative representations from raw observations, the abundance of data instances allows for generalised representations. In turn, generalised representations allow for effective learning of complex tasks. Despite valuable efforts in extraction of information from single data sources or data types, dealing with multiple diverse data sources remains an open question in the scientific community. Representation learning enables combination and juxtaposition of multiple diverse data sources in a meaningful, common and lower-dimensional space. However, typical learning frameworks for joint representation learning, face a plethora of challenges. Initially, architectural decisions of the involved neural networks is often a product of manual work or application specific decisions that rarely generalise to multiple domains or tasks. At the same time, directly tying data sources in the input layers of the neural network introduces an expectation of constant availability. However, expecting all data sources to be constantly available, is not realistic in real world applications. In addition, the involvment of redudant or non-complementary data sources may lead to detoriating performance. However, dealing with such sources requires manual effort. Such effort, is put into creating explicit assumptions or rules that will ensure stability, or to understand the intricate relations between data sources, in order to avoid non-complementary ones. This thesis includes the formulation and investigation of the hypothesis that external data evidence improves deep representation learning. The above investigation results in the proposal of a deep representation learning method, called Evidence Transfer (EviTraN). EviTraN is a versatile and automated fusion scheme based on deep representation learning, transfer learning and hybrid modelling between generative and discriminative views. In addition, it leads to the proposal of a set of evaluation criteria for deep representation learning for the purposes of information fusion. Furthermore, this thesis includes a theoretical interpretation of the above method, based on comparison with the well-received Information Bottleneck method, that acts as a stepping stone towards explainable modelling and open science. The evaluation process of EviTraN also includes a realistic scenario of detecting severe weather in an unsupervised manner. Thus, demonstrating its impact and potential use in additional real-world applications. Experimental evaluation with artificially generated, as well as, realistic evidence sources suggest that EviTraN is a robust and effective method. In addition, it is versatile, as it allows the introduction of a variety of relations, including non-complementary ones. Furthermore, due to its learning process based on the transfer learning setting, it is a modular fusion scheme that does not require all data sources to be present during inference (only the primary data instances).

Teaching algorithmic programming in secondary education through sound

Abstract: It is considered hard to teach programming in secondary education while achieving the aims of curriculum (several studies are presented in this research). However, when teaching is supported by suitable methodologies, learning can be ameliorated. Under this premise, the first part, chapter 1, which is the introductory chapter, describes the role and purpose of algorithmic programming in Secondary Education, as well as the research questions, the purpose and the structure of the dissertation. Chapter 2 includes a concise retrospection of the current situation of teaching tools. This thesis discusses different teaching approaches to programming in secondary (high-school) education and examines the potential benefit of sound-alerts as a complementary teaching tool in Chapters 3, 4 and 5. It is based on a methodology of participatory observation, experimentation, quantitative, qualitative analysis and evaluation.Furthermore a description, analysis, evaluation of proposed teaching scenarios and the students' conclusions in chapter 6, take place. Such alerts were created by pairing different sound stimuli to specific programming actions and operations, such as the successful assignment of the correct data type to a variable. Both the selection of the sound stimuli, as well as the potential impact of the use of sound alerts on programming where evaluated through subjective studies. Results showed that participants preferred synthesized to natural (pre-recorded) stimuli for all types of alerts. It was also revealed that users prefer sound-alerts associated to pending actions, errors, successful code execution, conditional statements and looping over alerts highlighting the step-by-step execution of the code. The test results showed that students understand both the meaning and the use of code commands more clearly when they use a sound-enriched programming environment instead of a conventional one.The thesis concludes with the presentation of the potential future evolution and exploitation of the proposed model in the Next Generation programming environmentsin Chapter 7, as well as with the evaluation of the more important results of the approach. These results are the motive for the creation of an audio and voice messages data base and a new comprehensive educational tool using sound, which will be a new teachers’ aid to be able to teach algorithmic programming more effectively to their students, enhance classroom interaction and serve to further research into the benefits of sound and psychoacoustic perception in adolescents.

Theoretical study and development of photonic devices and sensors

Abstract: In this thesis emerging photonic platforms based on either plasmonic structures or sensing schemes using functional polymeric materials are studied both theoretically and experimentally. The first studied structure is metal-coated optical fiber tips with integrated plasmonic slot nanoresonators (PSNRs). The guiding and modal properties of metal-coated optical fiber tips with embedded PSNRs are investigated through Finite Element Method (FEM) simulations towards the identification of their optimization parameters. It was found that the placement of a PSNR at the cut-off radius of a metal-coated fiber tip, where the group velocity tends to zero, leads to considerable intensity enhancement of the field confined beyond the diffraction limit. Maximum intensity enhancement of optimally placed PSNRs at different radii shows a linear dependence between excitation wavelength and radius, making it feasible to engineer the proper radius for a specific wavelength for maximum enhancement. The second studied plasmonic platform are metal tips which can offer high field intensity at the tip apex and high confinement on the nanoscale. The proposed platform based on hybrid composite glass metal microwires can offer robustness, ease of light coupling as well as continuous re-excitation of the plasmon modes due to repeated total internal reflection at the glass/air interface which can dramatically reduce the high losses induced by the metal core. An optimized fabrication process of high-quality all-fiber plasmonic tips by tapering such hybrid metal core/dielectric cladding microfibers is proposed and demonstrated experimentally. For this purpose the Plateau-Rayleigh instability in such hybrid fibers is theoretically investigated by inducing surface tension perturbations and by comparing them to the Tomotika instability theory. The continuous-core breakup time is calculated via FEM microfluidic simulations for different temperatures. The theoretical results are in close agreement with the experimental observations providing insight into the engineering of fibers, towards the development of plasmonic tips. Plasmonic tips were successfully demonstrated in a highly controllable manner, and their performance was related to simulation results predicting high field enhancement factors up to 10^5.Optical fibers coated with novel polymers are also studied towards biosensing applications. The sensing mechanism is based on the variations of the measured power at the distal end of the fiber due to the interaction between the fiber’s evanescent field and the changes induced to the polymeric film by the adsorption of the under study molecules. Two novel amphiphilic block copolymers, the cationic PMMA117-b-PDMAEMA16 and the cationic vinyl-sulfone functionalized PMMA117-b-P(DMAEMA17-VSTEMA2), having both hydrophobic poly(methyl methacrylate) (PMMA) and hydrophilic poly[2-(dimethylamino)ethyl methacrylate] (PDMAEMA) blocks have been designed and synthesized for efficient protein detection in photonic-based sensing. The presence of the cationic PDMAEMA block and the vinyl-sulfone double bonds led to reversible electrostatic binding of negatively charged proteins like bovine serum albumin (BSA) and non-reversible chemical binding by thiol-ene reactions with cysteine in proteins, respectively. The sensing properties of these materials were assessed and confirmed by ATR-FTIR analysis and by the characterization of fabricated sensing heads on silica optical fibers functionalized with suitably deposited overlayers. The sensing assessment revealed the requirements for deposited overlayer characteristics towards proteins' detection sensitivity and selectivity enhancement. The fabrication of cost-effective, polymer-based electrospun fluorescent fibrous grids and their evaluation as candidates for sensing is also reported. The formation of 3D grids can provide large interaction area with gas analytes and thus overcome quenching limitations induced by polymeric films, for more efficient sensing. Two different polymer-based electrospun fibers having fluorescent moieties were fabricated. The first was fabricated by a well-defined, methacrylic homopolymer functionalized with anthracene moieties as fluorescent elements that has been blended with a commercially available poly(methyl methacrylate) for the production of fluorescent electrospun polymer fibers. These materials have been evaluated for ammonia sensing based on the fluorescence quenching of the anthracene fluorophores in the presence of ammonia vapors, exhibiting fast response at concentration up to 10000 ppm. For the fabrication of the second fluorescent electrospun fiber system, ferrous core-shell nanoparticles consisting of a magnetic γ-Fe2O3 multi-nanoparticle core and an outer silica shell have been synthesized and covalently functionalized with Rhodamine B (RhB) fluorescent molecules (γ-Fe2O3/SiO2/RhB NPs). The resulting γ-Fe2O3/SiO2/RhB NPs were integrated with a renewable and naturally-abundant cellulose derivative (i.e. cellulose acetate, CA) that was processed in the form of electrospun fibers to yield multifunctional fluorescent fibrous nanocomposites. The encapsulation of the nanoparticles within the fibers and the covalent anchoring of the RhB fluorophore onto the nanoparticle surfaces prevented the fluorophore’s leakage from the fibrous mat, enabling thus stable fluorescence-based operation of the developed materials. These materials were further evaluated as dual fluorescent sensors (i.e. ammonia gas and pH sensors), demonstrating consistent response for very high ammonia concentrations (up to 12000 ppm) and fast and linear response in both alkaline and acidic environments. The superparamagnetic nature of embedded nanoparticles provides means of electrospun fibers morphology control by magnetic field-assisted processes and additional means of electromagnetic-based manipulation making possible their use in a wide range of sensing applications.

Scalable indexing and exploration of big time series data

Abstract: Time series are generated and stored at a vastly increasing rate in many industrial and research applications, including the Web and the Internet of Things, public utilities, finance, astronomy, biology, and many more. A significant portion concerns geolocated time series, i.e., those generated at, or otherwise associated with specific locations. Although several works have focused on efficient time series similarity search, there has been limited attention to the inherent challenge that geolocated time series introduce for hybrid queries, i.e., queries that involve both spatial proximity and time series similarity. Apart from traditional similarity search, we also consider the problem of detecting locally similar pairs and groups, called bundles, over co-evolving time series. These are pairs or groups of subsequences whose values do not differ by more than a predefined threshold for a number of consecutive timestamps. They could represent potentially valuable, concurrent common local patterns and trends among the time series. Time series visualization and visual analytics in general, is another field that has drawn the attention of the scientific community. However, there is a lack of efficient techniques for visual exploration and analysis of geolocated time series. Finally, large-scale time series forecasting has attracted a significant amount of interest, due to the highly complex nature of such data. In this thesis, we efficiently process hybrid queries through a hybrid index that we propose, called BTSR-tree. Furthermore, we address the problem of hybrid similarity joins over such geolocated time series. We introduce both centralized and MapReduce-based algorithms for performing such join operations using spatial-only, time series-only, and hybrid indices. Then, we tackle the problem of pair and bundle discovery over co-evolving time series, via a filter-verification technique that only examines candidate matches at judiciously chosen checkpoints across time. In the same line of work, we consider hybrid queries for retrieving geolocated time series based on filters that combine spatial distance and time series local similarity. To efficiently support such queries, we introduce the SBTSR-tree index, an extension of BTSR-tree that further optimizes local similarity search. Additionally, we present two approaches that rely on hybrid indices, allowing efficient map-based visual exploration and summarization of geolocated time series data. In particular, we use the BTSR-tree index and we introduce a new variant of the standard iSAX index, called geo-iSAX. We define the structure of the new index and show how both hybrid indices can be directly exploited to produce map-based visualizations of geolocated time series at different levels of granularity. Finally, towards large-scale time series forecasting, we introduce FML-kNN, a novel distributed processing framework for big data that performs probabilistic classification and regression. The framework’s core is consisted of a k-nearest neighbor joins algorithm which, contrary to similar approaches, is executed in a single distributed session and scales on very large volumes of data of variable granularity and dimensionality. Throughout this thesis, we experimentally and empirically evaluate our work using synthetic and real-world datasets from diverse domains, against baseline and state-of-the-art existing methods, demonstrating the efficiency and superiority of our approaches.

Abstract: The field of research that deals with the Spectral Estimation problem is fundamental and plays a pivotal role within the wider scientific area of signal processing. Even though this field has been the scope of considerable research activity for several decades, recently there has been increased interest from the scientific community regarding a specific category of Spectral Estimators, the so-called Adaptive Filter-Banks. This category of estimators falls within the class of Non Parametric methods for spectral estimation and comprises techniques that are known to produce high quality estimates. However, these methods are modeled by highly perplexed mathematical formulations which - unavoidably - require an extremely high computational load when implemented in a straightforward manner (brute force). This doctoral thesis attempts to contribute in this field of study, setting as its primary goal, the formulation of low complexity algorithms for a series of spectral estimation methods of the aforementioned category. The study focuses - mainly - on the Capon, APES and IAA methods (for MSC spectrum estimation) and forms a mathematical model for each one, which though mathematically equivalent to the brute force approach, its theoretical computational complexity is orders of magnitude lower. In addition, a novel MSC estimator is proposed, based on the SLIM method, while for the most recently introduced methods (IAA, SLIM), new estimation schemes are proposed in order to incorporate the processing of signals containing arbitrarily missing data samples (missing data case), as well as two-dimensional signals. For all the newly introduced techniques, efficient implementation algorithms are devised. Building upon the above implementations for the batch processing case, the focus of this study is then directed towards building spectral estimation techniques, that are able to process time-varying signals (time-adaptive algorithms). So, with the appropriate reformulation and adjustment of the IAA method, an estimator of MSC spectrum is built, suitable for time-varying signals (including the missing data case). The work on the time-varying signals is extended towards the area of the PSD spectrum estimation for the missing data case with concurrent implementation of a "recovery" mechanism for the missing samples. The proposed estimators operate by advancing the analysis window by one sample at a time or by blocks of samples. Moreover, the two dimensional time-varying signal hypothesis is also studied. Highly efficient algorithms are derived for all the time-varying (missing data) methods, that require far less computational effort compared to the fastest implementations of other high quality techniques. The performance of the aforementioned, proposed methods is illustrated via typical simulation experiments, while their theoretical complexity is calculated and depicted along with the complexity of the brute force implementations.