The first blot post on this series was focus on the data part. This one will be focused on the code and functionality part. After explaining the Strangler Pattern and understanding some of the options and the challenges, we will cover how we can combine Strangler and CDC as the ultimate solution for Slice and Dice the Monolith. Migrations are hard and often need to be performed into waves. There is a reason for that. Strangler is a code pattern focused on stopping the bleeding and either easily or more painfully migrating to a new codebase from either a legacy or a poorly designed solution. …
There are 3 reasons why these patterns might be beneficial and relevant for you. First of all, if you want to replicate a database because you want to run some analytical workload(I made a recent BigData post that might interest you and related to this topic). The second reason could that you want a new cluster or new database technology. The third use case can be you slice and dice you monolith, and you want to split one service into 2 or even more services. No matter your use cases, the architectural patterns I will describe in this blog post will help you when you need to migrate data at Scale. Dealing with data is not the same as dealing with services. Data migrations are much harder than service migrations. It’s pretty easy to do some rolling wave update and migrate service by service and re-deploy even reset them one by one. …
This is not a geography post. However, software architecture and code also suffer from erosion. People come and go, teams get created and destroyed by the project mindset. The result is an ownership problem. Ownership problem is just another form of a Governance issue; Governance is a bad world and often associated with bureaucracy and old ideas, and bad stuff. Ownership issues exist because teams touch a higher number of resources such as Code, Database Schemas, Dashboards, Alerts, Wiki pages, S3 Buckets, Kubernetes clusters, Ec2 instances, and much more assets. So why AWS TAGS are not enough?
The issues with…
So this is not a post about Slack(IRC/Chat application) — there is nothing about instant group messaging here. Slack also means LOOSE which is the opposite of tight. We all know Debit as bad. We all know Debit slows down execution and increases maintained cost. So why do we still suffer too much about it? IMHO there are several reasons that go from Bad management, Lack of Vision, Poor Tech Talent Pool, Lack of Software Ownership and Culture. Recently I finish Upstream from Dan Heath, which is an amazing book about product discovery but also about Software Architecture and Team Organization(Because both are the same :-) ). Dan has other amazing books like Switch. …
AI is here to stay. From recommendations to what book read next or what movie or tv show watch next to AI Art generated frames to Self-Driving cars. AI is a powerful technology that we do not master yet. It will bring changes for us as mankind which I’m not sure if we are ready for them. Currently, we have problems understanding WHY ai did something and debug and understand why some outcome happened. This not only is problematic to fight biases but also because we will be doing more complicated things with AI. Today I want to share a simple and short slidecast I made to get you thinking about this. This Slidecast was highly inspired by the books: Life 3.0 and HomoDeus. …
Documentation is always on the extremes. Why? Either a team has no documentation of what so ever or it has documentation paralysis and document every single aspect of the software development process to a point to have waste and slowness and little or no value. IMHO Documentation is part of that group of bad wards like Governance. The reality is Documentation is not what you think it is. There are several ways to document and might not even require to write anything down. Documentation is important. Often people dont know how to deal effectively with documentation and there are so many ways we can extract value from it. So value really depends on the kind of problem we are trying to solve. So let’s stop talking about documentation for a moment and let’s list some of the problems a software architect needs to help to FIX and then we can revisit documentation later with a fresh problem space lens. …
Software architecture it’s much more than satisfying requirements. Requirements can be tricky and several times misleading. Requirements are easy answers and they often shut down important discovery processes. Requirements are old as snakes, they were heavily praised by RUP, CMMI, PMI and still in vogue nowadays. The requirements list is just a wish list someone made. Requirements gave the false sensation of doing the right thing but this is just a lie. Requirements are often lying. Software architecture is a continuous process that requires thinking, judgment, leadership, and critique. Requirements are some easy to be gamed and often they are. So why people do it? Well, I think thats the wrong question. …
Dependencies are natural in big software solutions. However, dealing effectively with dependencies is critical in order to scale and reduce maintenance costs. Team dependencies are not a simple problem so will not be fixed with one simple answer. This is an old problem with most organizations suffers from it either in different degrees or different maturities but the problem is pretty much the same. Architecture is more than technology. Architecture and team organization work closely together and together can provide important mechanisms to scale, maintain, and better deal with team dependencies at Scale.
The Video
The Slides
Cheers,
Diego Pacheco
Originally published at http://diego-pacheco.blogspot.com on January 19, 2021.
Dealing with Tests Dependencies is a way to reduce internal complexity and make services easier to maintain and more resilient to migrations and less expensive over time. Several times fixing tests is more expensive than fixing the real code itself. That could happen for several reasons either by lack of trust between teams or because there is a lack of ways to deal with test dependencies issues. Often people don’t pay much attention to tests because either is an engineering or QA problem. However, tests are critical for proper CI/CD and at scale require attention. Overtime complexity and tech debt can charge a high price making all future improvements harder, thats why we need to have ways to deal with this problem in the most effective way possible. …
Yes, Big Data is hard. It’s not hard only because of the number of technologies a good data engineering team needs to master technologies such as Spark, Flink, and Kafka Streams(Batch and Streaming), Hadoop, HDFS, and Hive if you have a DW legacy(most likely you do) and the Data Science part of it with Discovery and Execution at Scale. There are needs for different kinds of storage and Design/Modeling, and thats, not even the hard part. The technology landscape gets bigger and bigger as time pass. We have many specializations such as Frontend/Mobile engineering, Backend Engineering, Architecture, DevOps(Which is a movement, not a department, but all companies decide is a role, so you know what I mean), QA(a dying one?), Product, Management and Data Engineering which often has Data Scientists working with Data Engineers. To some degree, Data Engineering and Data Science have the same issues as Product has today. …
About