What is Google Cloud Bigtable and what are its top alternatives?
Google Cloud Bigtable is a fully managed, scalable NoSQL database service designed for large analytical and operational workloads. Key features include automatic scalability, low latency, high throughput, and seamless integration with other Google Cloud services. However, some limitations include high pricing for smaller workloads and a steep learning curve for users unfamiliar with NoSQL databases.
- Amazon DynamoDB: Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. Key features include single-digit millisecond latency, automated backups, and built-in security features. Pros: easy scalability, seamless integration with AWS services. Cons: pricing can be expensive for large workloads.
- Azure Cosmos DB: Azure Cosmos DB is a globally distributed, multi-model database service that offers low latency and high availability. Key features include multiple data models, automatic scaling, and guaranteed high throughput. Pros: global distribution, multiple APIs supported. Cons: can be complex to manage for beginners.
- Cassandra: Apache Cassandra is an open-source, scalable, distributed database management system designed to handle large amounts of data with high availability and fault tolerance. Key features include linear scalability, decentralized architecture, and tunable consistency. Pros: decentralized architecture, fault tolerance. Cons: complex setup and maintenance.
- ScyllaDB: ScyllaDB is a high-performance, distributed database that is compatible with Apache Cassandra but delivers significantly higher throughput and lower latency. Key features include scale-out architecture, CQL compatibility, and real-time visibility into cluster performance. Pros: high throughput, low latency. Cons: limited support for certain data models.
- HBase: Apache HBase is an open-source, distributed, scalable, and consistent database built on top of the Hadoop Distributed File System (HDFS). Key features include linear and modular scalability, high availability, and integration with Apache Hadoop. Pros: strong consistency, seamless integration with Hadoop ecosystem. Cons: limited support for ad-hoc queries.
- Redis: Redis is an open-source, in-memory data structure store that can be used as a database, cache, and message broker. Key features include high performance, support for various data structures, and built-in replication and clustering. Pros: high performance, versatile data structures. Cons: limited durability compared to disk-based databases.
- CockroachDB: CockroachDB is a distributed SQL database that is designed for global cloud services with strong consistency, ultra-resilience, and scalability. Key features include automatic scaling, geo-partitioning, and strong consistency guarantees. Pros: strong consistency, global scalability. Cons: can be complex to manage due to distributed nature.
- Aerospike: Aerospike is a high-performance, distributed NoSQL database built for speed at scale with a unique hybrid memory architecture. Key features include low-latency reads and writes, automatic data balancing, and strong consistency. Pros: low latency, high throughput. Cons: limited query capabilities compared to other databases.
- MarkLogic: MarkLogic is a multi-model NoSQL database platform that allows users to store, manage, and search structured and unstructured data. Key features include ACID transactions, versatile indexing capabilities, and semantic search. Pros: multi-model support, semantic search. Cons: can be complex to set up and configure for specific use cases.
- Cockroach Cloud: Cockroach Cloud is a fully managed, geo-distributed database service built on top of CockroachDB that offers automated scaling, high availability, and ultra-resilience. Key features include automatic failover, geo-replication, and horizontal scalability. Pros: fully managed service, global distribution. Cons: pricing can be expensive for large workloads.
Top Alternatives to Google Cloud Bigtable
- Google Cloud Datastore
Use a managed, NoSQL, schemaless database for storing non-relational data. Cloud Datastore automatically scales as you need it and supports transactions as well as robust, SQL-like queries. ...
- Microsoft Access
It is an easy-to-use tool for creating business applications, from templates or from scratch. With its rich and intuitive design tools, it can help you create appealing and highly functional applications in a minimal amount of time. ...
- Google Cloud Spanner
It is a globally distributed database service that gives developers a production-ready storage solution. It provides key features such as global transactions, strongly consistent reads, and automatic multi-site replication and failover. ...
- MongoDB
MongoDB stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema. MongoDB was also designed for high availability and scalability, with built-in replication and auto-sharding. ...
- Google Cloud Storage
Google Cloud Storage allows world-wide storing and retrieval of any amount of data and at any time. It provides a simple programming interface which enables developers to take advantage of Google's own reliable and fast networking infrastructure to perform data operations in a secure and cost effective manner. If expansion needs arise, developers can benefit from the scalability provided by Google's infrastructure. ...
- Google Cloud SQL
Run the same relational databases you know with their rich extension collections, configuration flags and developer ecosystem, but without the hassle of self management. ...
- JavaScript
JavaScript is most known as the scripting language for Web pages, but used in many non-browser environments as well such as node.js or Apache CouchDB. It is a prototype-based, multi-paradigm scripting language that is dynamic,and supports object-oriented, imperative, and functional programming styles. ...
- Git
Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. ...
Google Cloud Bigtable alternatives & related posts
- High scalability7
- Serverless2
- Ability to query any property2
- Pay for what you use1
related Google Cloud Datastore posts
related Microsoft Access posts
Google Cloud Spanner
- Strongly consistent1
- Horizontal scaling1
- Scalable1
related Google Cloud Spanner posts
- Document-oriented storage827
- No sql593
- Ease of use553
- Fast464
- High performance410
- Free257
- Open source218
- Flexible180
- Replication & high availability145
- Easy to maintain112
- Querying42
- Easy scalability39
- Auto-sharding38
- High availability37
- Map/reduce31
- Document database27
- Easy setup25
- Full index support25
- Reliable16
- Fast in-place updates15
- Agile programming, flexible, fast14
- No database migrations12
- Easy integration with Node.Js8
- Enterprise8
- Enterprise Support6
- Great NoSQL DB5
- Support for many languages through different drivers4
- Drivers support is good3
- Aggregation Framework3
- Schemaless3
- Fast2
- Managed service2
- Easy to Scale2
- Awesome2
- Consistent2
- Good GUI1
- Acid Compliant1
- Very slowly for connected models that require joins6
- Not acid compliant3
- Proprietary query language1
related MongoDB posts
I just finished the very first version of my new hobby project: #MovieGeeks. It is a minimalist online movie catalog for you to save the movies you want to see and for rating the movies you already saw. This is just the beginning as I am planning to add more features on the lines of sharing and discovery
For the #BackEnd I decided to use Node.js , GraphQL and MongoDB:
Node.js has a huge community so it will always be a safe choice in terms of libraries and finding solutions to problems you may have
GraphQL because I needed to improve my skills with it and because I was never comfortable with the usual REST approach. I believe GraphQL is a better option as it feels more natural to write apis, it improves the development velocity, by definition it fixes the over-fetching and under-fetching problem that is so common on REST apis, and on top of that, the community is getting bigger and bigger.
MongoDB was my choice for the database as I already have a lot of experience working on it and because, despite of some bad reputation it has acquired in the last months, I still believe it is a powerful database for at least a very long list of use cases such as the one I needed for my website
I am starting to become a full-stack developer, by choosing and learning .NET Core for API Development, Angular CLI / React for UI Development, MongoDB for database, as it a NoSQL DB and Flutter / React Native for Mobile App Development. Using Postman, Markdown and Visual Studio Code for development.
- Scalable28
- Cheap19
- Reliable14
- Easy9
- Chealp3
- More praticlal and easy1
related Google Cloud Storage posts
Context: I wanted to create an end to end IoT data pipeline simulation in Google Cloud IoT Core and other GCP services. I never touched Terraform meaningfully until working on this project, and it's one of the best explorations in my development career. The documentation and syntax is incredibly human-readable and friendly. I'm used to building infrastructure through the google apis via Python , but I'm so glad past Sung did not make that decision. I was tempted to use Google Cloud Deployment Manager, but the templates were a bit convoluted by first impression. I'm glad past Sung did not make this decision either.
Solution: Leveraging Google Cloud Build Google Cloud Run Google Cloud Bigtable Google BigQuery Google Cloud Storage Google Compute Engine along with some other fun tools, I can deploy over 40 GCP resources using Terraform!
Check Out My Architecture: CLICK ME
Check out the GitHub repo attached
In #Aliadoc, we're exploring the crowdfunding option to get traction before launch. We are building a SaaS platform for website design customization.
For the Admin UI and website editor we use React and we're currently transitioning from a Create React App setup to a custom one because our needs have become more specific. We use CloudFlare as much as possible, it's a great service.
For routing dynamic resources and proxy tasks to feed websites to the editor we leverage CloudFlare Workers for improved responsiveness. We use Firebase for our hosting needs and user authentication while also using several Cloud Functions for Firebase to interact with other services along with Google App Engine and Google Cloud Storage, but also the Real Time Database is on the radar for collaborative website editing.
We generally hate configuration but honestly because of the stage of our project we lack resources for doing heavy sysops work. So we are basically just relying on Serverless technologies as much as we can to do all server side processing.
Visual Studio Code definitively makes programming a much easier and enjoyable task, we just love it. We combine it with Bitbucket for our source code control needs.
Google Cloud SQL
- Fully managed13
- Backed by Google10
- SQL10
- Flexible4
- Encryption at rest and transit3
- Automatic Software Patching3
- Replication across multiple zone by default3
related Google Cloud SQL posts
We use Go for the first-off due to our knowledge of it. Second off, it's highly performant and optimized for scalability. We run it using dockerized containers for our backend REST APIs.
For Frontend, we use React with Next.js at vercel. We use NextJS here mostly due to our need for Server Side Rendering and easier route management.
For Database, we use MySQL as it is first-off free and always has been in use with us. We use Google Cloud SQL from GCP that manages its storage and versions along with HA.
All stacks are free to use and get the best juice out of the system. We also use Redis for caching for enterprise-grade apps where data retrieval latency matters the most.
As far as the backend goes, we first had to decide which database will power most of Daily services. Considering relational databases vs document datbases, we decided that the relational model is a better fit for Daily as we have a lot of connections between the different entities. At the time MySQL was the only service available on Google Cloud SQL so this was out choice. In terms of #backend development Node.js powers most of our services, thanks to its amazing ecosystem there are a lot of modules publicly available to shorten the development time. Go is for the light services which are all about performance and delivering quickly the response, such as our redirector service.
JavaScript
- Can be used on frontend/backend1.7K
- It's everywhere1.5K
- Lots of great frameworks1.2K
- Fast896
- Light weight745
- Flexible425
- You can't get a device today that doesn't run js392
- Non-blocking i/o286
- Ubiquitousness236
- Expressive191
- Extended functionality to web pages55
- Relatively easy language49
- Executed on the client side46
- Relatively fast to the end user30
- Pure Javascript25
- Functional programming21
- Async15
- Full-stack13
- Setup is easy12
- Its everywhere12
- Future Language of The Web12
- JavaScript is the New PHP11
- Because I love functions11
- Like it or not, JS is part of the web standard10
- Expansive community9
- Everyone use it9
- Can be used in backend, frontend and DB9
- Easy9
- Easy to hire developers8
- No need to use PHP8
- For the good parts8
- Can be used both as frontend and backend as well8
- Powerful8
- Most Popular Language in the World8
- Popularized Class-Less Architecture & Lambdas7
- It's fun7
- Nice7
- Versitile7
- Hard not to use7
- Its fun and fast7
- Agile, packages simple to use7
- Supports lambdas and closures7
- Love-hate relationship7
- Photoshop has 3 JS runtimes built in7
- Evolution of C7
- 1.6K Can be used on frontend/backend6
- Client side JS uses the visitors CPU to save Server Res6
- It let's me use Babel & Typescript6
- Easy to make something6
- Can be used on frontend/backend/Mobile/create PRO Ui6
- Promise relationship5
- Stockholm Syndrome5
- Function expressions are useful for callbacks5
- Scope manipulation5
- Everywhere5
- Client processing5
- Clojurescript5
- What to add5
- Because it is so simple and lightweight4
- Only Programming language on browser4
- Test21
- Easy to learn1
- Easy to understand1
- Not the best1
- Hard to learn1
- Subskill #41
- Test1
- Hard 彤0
- A constant moving target, too much churn22
- Horribly inconsistent20
- Javascript is the New PHP15
- No ability to monitor memory utilitization9
- Shows Zero output in case of ANY error8
- Thinks strange results are better than errors7
- Can be ugly6
- No GitHub3
- Slow2
related JavaScript posts
Oof. I have truly hated JavaScript for a long time. Like, for over twenty years now. Like, since the Clinton administration. It's always been a nightmare to deal with all of the aspects of that silly language.
But wowza, things have changed. Tooling is just way, way better. I'm primarily web-oriented, and using React and Apollo together the past few years really opened my eyes to building rich apps. And I deeply apologize for using the phrase rich apps; I don't think I've ever said such Enterprisey words before.
But yeah, things are different now. I still love Rails, and still use it for a lot of apps I build. But it's that silly rich apps phrase that's the problem. Users have way more comprehensive expectations than they did even five years ago, and the JS community does a good job at building tools and tech that tackle the problems of making heavy, complicated UI and frontend work.
Obviously there's a lot of things happening here, so just saying "JavaScript isn't terrible" might encompass a huge amount of libraries and frameworks. But if you're like me, yeah, give things another shot- I'm somehow not hating on JavaScript anymore and... gulp... I kinda love it.
How Uber developed the open source, end-to-end distributed tracing Jaeger , now a CNCF project:
Distributed tracing is quickly becoming a must-have component in the tools that organizations use to monitor their complex, microservice-based architectures. At Uber, our open source distributed tracing system Jaeger saw large-scale internal adoption throughout 2016, integrated into hundreds of microservices and now recording thousands of traces every second.
Here is the story of how we got here, from investigating off-the-shelf solutions like Zipkin, to why we switched from pull to push architecture, and how distributed tracing will continue to evolve:
https://eng.uber.com/distributed-tracing/
(GitHub Pages : https://www.jaegertracing.io/, GitHub: https://github.com/jaegertracing/jaeger)
Bindings/Operator: Python Java Node.js Go C++ Kubernetes JavaScript OpenShift C# Apache Spark
- Distributed version control system1.4K
- Efficient branching and merging1.1K
- Fast959
- Open source845
- Better than svn726
- Great command-line application368
- Simple306
- Free291
- Easy to use232
- Does not require server222
- Distributed27
- Small & Fast22
- Feature based workflow18
- Staging Area15
- Most wide-spread VSC13
- Role-based codelines11
- Disposable Experimentation11
- Frictionless Context Switching7
- Data Assurance6
- Efficient5
- Just awesome4
- Github integration3
- Easy branching and merging3
- Compatible2
- Flexible2
- Possible to lose history and commits2
- Rebase supported natively; reflog; access to plumbing1
- Light1
- Team Integration1
- Fast, scalable, distributed revision control system1
- Easy1
- Flexible, easy, Safe, and fast1
- CLI is great, but the GUI tools are awesome1
- It's what you do1
- Phinx0
- Hard to learn16
- Inconsistent command line interface11
- Easy to lose uncommitted work9
- Worst documentation ever possibly made7
- Awful merge handling5
- Unexistent preventive security flows3
- Rebase hell3
- When --force is disabled, cannot rebase2
- Ironically even die-hard supporters screw up badly2
- Doesn't scale for big data1
related Git posts
Our whole DevOps stack consists of the following tools:
- GitHub (incl. GitHub Pages/Markdown for Documentation, GettingStarted and HowTo's) for collaborative review and code management tool
- Respectively Git as revision control system
- SourceTree as Git GUI
- Visual Studio Code as IDE
- CircleCI for continuous integration (automatize development process)
- Prettier / TSLint / ESLint as code linter
- SonarQube as quality gate
- Docker as container management (incl. Docker Compose for multi-container application management)
- VirtualBox for operating system simulation tests
- Kubernetes as cluster management for docker containers
- Heroku for deploying in test environments
- nginx as web server (preferably used as facade server in production environment)
- SSLMate (using OpenSSL) for certificate management
- Amazon EC2 (incl. Amazon S3) for deploying in stage (production-like) and production environments
- PostgreSQL as preferred database system
- Redis as preferred in-memory database/store (great for caching)
The main reason we have chosen Kubernetes over Docker Swarm is related to the following artifacts:
- Key features: Easy and flexible installation, Clear dashboard, Great scaling operations, Monitoring is an integral part, Great load balancing concepts, Monitors the condition and ensures compensation in the event of failure.
- Applications: An application can be deployed using a combination of pods, deployments, and services (or micro-services).
- Functionality: Kubernetes as a complex installation and setup process, but it not as limited as Docker Swarm.
- Monitoring: It supports multiple versions of logging and monitoring when the services are deployed within the cluster (Elasticsearch/Kibana (ELK), Heapster/Grafana, Sysdig cloud integration).
- Scalability: All-in-one framework for distributed systems.
- Other Benefits: Kubernetes is backed by the Cloud Native Computing Foundation (CNCF), huge community among container orchestration tools, it is an open source and modular tool that works with any OS.
Often enough I have to explain my way of going about setting up a CI/CD pipeline with multiple deployment platforms. Since I am a bit tired of yapping the same every single time, I've decided to write it up and share with the world this way, and send people to read it instead ;). I will explain it on "live-example" of how the Rome got built, basing that current methodology exists only of readme.md and wishes of good luck (as it usually is ;)).
It always starts with an app, whatever it may be and reading the readmes available while Vagrant and VirtualBox is installing and updating. Following that is the first hurdle to go over - convert all the instruction/scripts into Ansible playbook(s), and only stopping when doing a clear vagrant up
or vagrant reload
we will have a fully working environment. As our Vagrant environment is now functional, it's time to break it! This is the moment to look for how things can be done better (too rigid/too lose versioning? Sloppy environment setup?) and replace them with the right way to do stuff, one that won't bite us in the backside. This is the point, and the best opportunity, to upcycle the existing way of doing dev environment to produce a proper, production-grade product.
I should probably digress here for a moment and explain why. I firmly believe that the way you deploy production is the same way you should deploy develop, shy of few debugging-friendly setting. This way you avoid the discrepancy between how production work vs how development works, which almost always causes major pains in the back of the neck, and with use of proper tools should mean no more work for the developers. That's why we start with Vagrant as developer boxes should be as easy as vagrant up
, but the meat of our product lies in Ansible which will do meat of the work and can be applied to almost anything: AWS, bare metal, docker, LXC, in open net, behind vpn - you name it.
We must also give proper consideration to monitoring and logging hoovering at this point. My generic answer here is to grab Elasticsearch, Kibana, and Logstash. While for different use cases there may be better solutions, this one is well battle-tested, performs reasonably and is very easy to scale both vertically (within some limits) and horizontally. Logstash rules are easy to write and are well supported in maintenance through Ansible, which as I've mentioned earlier, are at the very core of things, and creating triggers/reports and alerts based on Elastic and Kibana is generally a breeze, including some quite complex aggregations.
If we are happy with the state of the Ansible it's time to move on and put all those roles and playbooks to work. Namely, we need something to manage our CI/CD pipelines. For me, the choice is obvious: TeamCity. It's modern, robust and unlike most of the light-weight alternatives, it's transparent. What I mean by that is that it doesn't tell you how to do things, doesn't limit your ways to deploy, or test, or package for that matter. Instead, it provides a developer-friendly and rich playground for your pipelines. You can do most the same with Jenkins, but it has a quite dated look and feel to it, while also missing some key functionality that must be brought in via plugins (like quality REST API which comes built-in with TeamCity). It also comes with all the common-handy plugins like Slack or Apache Maven integration.
The exact flow between CI and CD varies too greatly from one application to another to describe, so I will outline a few rules that guide me in it: 1. Make build steps as small as possible. This way when something breaks, we know exactly where, without needing to dig and root around. 2. All security credentials besides development environment must be sources from individual Vault instances. Keys to those containers should exist only on the CI/CD box and accessible by a few people (the less the better). This is pretty self-explanatory, as anything besides dev may contain sensitive data and, at times, be public-facing. Because of that appropriate security must be present. TeamCity shines in this department with excellent secrets-management. 3. Every part of the build chain shall consume and produce artifacts. If it creates nothing, it likely shouldn't be its own build. This way if any issue shows up with any environment or version, all developer has to do it is grab appropriate artifacts to reproduce the issue locally. 4. Deployment builds should be directly tied to specific Git branches/tags. This enables much easier tracking of what caused an issue, including automated identifying and tagging the author (nothing like automated regression testing!).
Speaking of deployments, I generally try to keep it simple but also with a close eye on the wallet. Because of that, I am more than happy with AWS or another cloud provider, but also constantly peeking at the loads and do we get the value of what we are paying for. Often enough the pattern of use is not constantly erratic, but rather has a firm baseline which could be migrated away from the cloud and into bare metal boxes. That is another part where this approach strongly triumphs over the common Docker and CircleCI setup, where you are very much tied in to use cloud providers and getting out is expensive. Here to embrace bare-metal hosting all you need is a help of some container-based self-hosting software, my personal preference is with Proxmox and LXC. Following that all you must write are ansible scripts to manage hardware of Proxmox, similar way as you do for Amazon EC2 (ansible supports both greatly) and you are good to go. One does not exclude another, quite the opposite, as they can live in great synergy and cut your costs dramatically (the heavier your base load, the bigger the savings) while providing production-grade resiliency.