Federating Data Between Snowflake and Databricks with DuckDB and Apache Iceberg If you're running both Snowflake and Databricks — and most enterprises I work with are — you've probably hit the federation problem. Data lives in both platforms, analysts need to query across them, and the obvious solutions (ETL everything into one place, or pay… Continue reading Ducks on Icebergs
The rule of law is a fundamental principle from the Mesopotanian Code Ur-Nammu, through Magna Carta to International Criminal Court's decisiion to ditch Microsoft Office for European open source alternatives. Data sovereignty requires certainty that services will never be terminated or at the mastery of a governmental body. For this reason I find it useful… Continue reading EU Sovereign Cloud List
Faster Insight, Better Reuse, and Scalable Data Foundations Healthcare organisations face growing demand for better use of data: improving operational performance, supporting population health management, enabling AI, and accelerating research. Yet many still rely on fragmented pipelines, duplicated transformations, and slow bespoke data requests. At the same time, the economics of technology have changed. Modern… Continue reading Explaining Medallion Data Architectures in Healthcare
It was a pleasure to be invited to speak again at Google's Geo for Good event in Mountain View in October 2023. My talk covered Nature Metrics actionable nature intelligence for the future and the large scale machine learning work we are doing for Species Distribution Modelling. https://youtu.be/E3V6ePXRT-w?list=PLLW-qoCMKQsx_YHs95aOHguU7XpKl75Fk&t=2439
My presentation on Biodiversity monitoring at Google's Geo for Good summit Sep 22
Introducing A New Organisation to Agile Product Development As the new CTO, Chief Product Owner, Chief Architect, Lead Engineer or just natural leader it may be your responsibility to introduce an Agile way of working to a new Organisation. We are going to walk through the common steps to successfully introduce Agile methodology and we… Continue reading The Agile Architecture Guide (part 1: Product Development)
The LAPSUS$ cybercrime group which deleted 50TB of patient data from Brazil’s Ministry of Health have this week disclosed breaches on both Microsoft and Okta. LAPSUS$ are threatening to publish leaked data from Microsoft (source code) and Okta (clients) unless a ransom is paid. LAPSUS$ claim NVIDIA, Samsung, and Vodafone as targets that they have… Continue reading SIM Swapping Prevention
Product Management as a discipline sits at the intersection of business need, user experience and technology. A consistent Product Management strategy is necessary for delivering a viable and sustainable product. When the Product Management strategy deviates with every Product Manager hire then the focus and investment can become confused, and you end up a bit… Continue reading AI Product Management as a Markov Chain
Medical data ownership needs to pass from the trust to the patient to enable ML diagnostics and better research collaboration. The best way of achieving this without being compelled by the hyper-scalers is to move to a persistent container based architecture.
The following is my costed solution architecture for delivering an equivalent to Operation Moonshot based on RT-LAMP testing and experience from the Francis Crick Institute
There have been 10 UK Government U-Turns so far in 2020. Each change will have had an associated IT change cost. This is my best personal assessment of what each of these changes would likely have cost. I will provide justification for each of my assumptions and will tend towards a lower possible range. I… Continue reading IT Spend Analysis of UK Government U-Turns in 2020 (so far)
I really enjoy working at the Francis Crick Institute and I am really proud to have worked on the SARS-CoV-2 Testing Service. It's all quite different from mobile networks Find out how @TheCrick worked with @uclh NHS Foundation Trust and Health Services Laboratories to set up a SARS-CoV-2 testing service in just two weeks that… Continue reading SARS-CoV-2 Testing Service
The UK is trialling its Covid-19 contact tracing application which tracks human interactions. The app uses Bluetooth Low Energy (BLE) communications between smartphones for registering handshakes' duration and distance. This data is then uploaded to a centralised database so that if a user self-registers as Covid-19 positive, the centralised service can push notifications to all… Continue reading How do Covid-19 tracing apps work and how do they comply with EU-GDPR?
The Francis Crick Institute has repurposed its laboratories as an emergency Covid-19 testing facility. The Crick is helping combat the spread of infection and allow key workers to perform lifesaving duties and remain safe. Crick repurposed PCR testing lab One of the main technologies the Crick is using in this effort are Polymerase Chain Reaction… Continue reading Unit Tests for Covid-19 Polymerase Chain Reaction Software
My personal experiences of working with Huawei at the UK's largest mobile operator.
Mobile Network Operators have an opportunity to provide a secure UK Open Banking Service using their location services, 5G Network Slicing and Password-less Multi-Factor Authentication
As a Google Cloud Platform certified architect I really should blog some more about my actual usage of GCP. One of my favourite tools is Dataproc as it provides a managed Spark & Hadoop environment and enables a lambda architecture suitable for complex network event processing and function remediation. A mobile radio network is a… Continue reading Telco Use Cases for Google Cloud Dataproc – managed Spark & Hadoop for Mobile Network Performance Data
The City of Sacramento has deployed 300+ small cells as part of a 5G Fixed Wireless Access deployment with Verizon. These deployments can only provide partial 5G coverage of a city like Sacramento. This is because of the relative transmission range of 26Ghz and 28Ghz spectrum. In fill is required with further small cells and… Continue reading Guaranteeing Network Slices: The Role of 5G WAN Optimisation for Small Cells
API Crazy 