Recently, I was working on a challenge posted by A Cloud Guru where we need to process Covid19 data from two sources and store it into a database for further reporting. Here are the approach and difficulties I faced during the development and testing phase

Approach:

Data Analysis: Being a data processing project, I first wanted to start exploring the data and the Jupyter notebook is the best tool for this purpose. I loaded the into Jypyter notebook and with help of NumPy and Pandas, I was able to check out columns in files and its data types. …


In this blog, we would be talking about how we can implement Single Sing-On for AWS Console and AWS CLI using a hybrid solution. As we know, Azure AD is a fully managed identity service for SSO. Its conditional access policy and multifactor authentication help to protect user access.

Let's talk a little bit about the services we are going to use in this project.

AWS SSO: It helps the organization to create centrally managed user access to multiple accounts and applications with a single sign-on. There are three ways to create an Identity store in AWS SSO.

  1. AWS SSO itself can act as an Identity Store. You can create/manage users and groups. …


In this post, we would be solving two problems. The first one is “How to create CI/CD pipeline for Hashicorp Packer AMI builder” and the second one is “How to automatically refresh AWS Autoscaling Group instances automatically without any downtime.”

Let's talk about the scenario which I have created to demonstrate the solution for the above two problems. So, we would be creating a simple web server based on Apache HTTPd service. It will be highly available using the Autoscaling group backed by Launch Template and load balanced with help of Application Load Balancer. …


In general, when we want to create a large Web Application that is fast, scalable, and API based then we can use React with GraphQL( with Appolo client). It takes huge effort to build such large web projects. In this blog post, we will talk about how we can do such projects cloud-native way on React and GraphQL which will take off some of the repetitive works in the traditional approach.

So what are the services we are going to use here from AWS?

  1. AWS AppSync for GraphQL API
  2. AWS Amplify for web apps
  3. AWS DynamoDB for NoSQL DB
  4. AWS Cognito for User…

I have followed up MongoDB from past 6 month or so. Recently, Mongo has released its 4.0 version. So it was again an opportunity for me to visit Mongo University to grab latest updates. Last week, I finished the course and I have to admit that it has really very cool updates which will surly make MongoDB stand out in comparison with other databases in long run. It was a huge release with over 100 new features.

Let`s start and go through few important updates.

ACID Transaction:

MongoDB has extended its ACID transaction functionality superbly. A transaction can now support to multiple statements and multiple documents across one or multiple collections. However, multi document transactions work over replica sets only. So, developers need not to think about limiting their logic to single document. By default, if a transaction runs more than 60 seconds, it will be automatically rolled back by WiredTiger engine and an exception will be returned back to the driver. There is no limit on number of documents that can be processed in single transaction however MongoDB recommend to limit it around 1000 documents per transaction. Transaction will be automatically aborted in case of cache pressure on server. Write conflicts have been taken care properly to ensure consistency and integrity. Transient transaction error exception can be observed in the case of write happened before write lock was taken but not present into transaction snapshot. …

About

Ankit K Singh

A data science enthusiast and Passionate for Cloud Technology. AWS, Azure, and MongoDB certified developer/architect. Interested in Google Cloud Engineering.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store