Understanding gRPC in the dawn of microservices.
Microservices are small programs, each with a specific and narrow scope, that are glued together to produce what appears from the outside to be one coherent web application. This architectural style is used in contrast with a traditional "monolith" where every component and sub-routine of the application is bundled into one codebase and not separated by a network boundary. In recent years microservices have enjoyed increased popularity, concurrent with (but not necessarily requiring the use of) enabling new technologies such as Amazon Web Services
. In this article, we will take a look at the "what" and "why" of microservices and at gRPC, an open source framework released by Google, which is a tool organizations are increasingly reaching for in their migration towards microservices.
Why Use Microservices?
To understand the general history and structure of microservices emerging as an architectural pattern, this
Continue reading "What’s in a transport layer?"
A deep dive into Uber's engineering effort to optimize geospatial queries in Presto.
From determining the most convenient rider pickup points to predicting the fastest routes, Uber aims to use data-driven analytics to create seamless trip experiences. Within engineering, analytics inform decision-making processes across the board.
One of the distinct challenges for Uber is analyzing geospatial big data. City locations, trips, and event information, for instance, provide insights that can improve business decisions and better serve users. Geospatial data analysis is particularly challenging, especially in a big data scenario, such as computing how many rides start at a transit location, how many drivers are crossing state lines, and so on. For these analytical requests, we must achieve efficiency, usability, and scalability in order to meet user needs and business requirements.
To accomplish this, we use Presto
in our production environment to process the big data powering our interactive SQL engine.
Continue reading "Query the planet: Geospatial big data analytics at Uber"
Building convnets from scratch with TensorFlow and TensorBoard.
Given all of the higher level tools that you can use with TensorFlow, such as tf.contrib.learn
, one can very easily build a convolutional neural network with a very small amount of code. But often with these higher level applications, you cannot access the little inbetween bits of the code, and some of the understanding of what’s happening under the surface is lost.
In this tutorial, I’ll walk you through how to build a convolutional neural network from scratch, using just the low-level TensorFlow and visualizing our graph and network performance using TensorBoard. If you don't understand some of the basics of a fully connected neural network, I highly recommend you first check out Not another MNIST tutorial with TensorFlow
. Throughout this article, I will also break down each step of the convolutional neural network to its absolute
Continue reading "Visualizing convolutional neural networks"
The O'Reilly Podcast: Dave Cassel on building a unified enterprise database to store and query any type of data.
In this podcast episode, I speak with Dave Cassel, technical community manager at MarkLogic, creator of a multi-model NoSQL database that aims to integrate data silos for a unified view. We talked about integration patterns for loading and exporting data at ease, an architecture that enables efficient search and queries, and layers of security that follow the data from its original source throughout its lifecycle.
Work on applications, as soon as you load the data
The idea of 'load as-is' is that your data already exists in some form, and that form can vary dramatically. It can be word documents or XML or JSON data. It can also be stuff that you've already got in relational databases. The idea here is that if we can take that data in whatever form Continue reading "Load, search, and secure data in multiple formats"