Transform 2022 will be back in person July 19, and virtually July 20 – 28, we are thrilled to announce. Join AI and data leaders for informative talks and exciting networking opportunities. Register now!
As a key driver of environmental changes, carbon emissions are increasingly being scrutinized by government regulators as well as the court of investor opinions. Recent Biden administration moves to limit greenhouse gassesAnd by the SEC to require public companies to disclose even very low carbon footprint impactsThey have received significant media attention, reporting on compliance trends that are only going to increase as the effects of climate changes become more obvious.
Two of the most popular public blockchains, Bitcoin, and Ethereum, use a proof-of work algorithm. This consumes a lot of processing powerBitcoin alone consumes around 136 Terawatt hours of electricity per annum, more than either Argentina or the Netherlands. These public chains are extremely inefficient per transaction and their power-hungry algorithms have inevitably led block construction to be known as Mining migrating to countries that have weaker environment laws and produce electricity from dirty sources like coal. This environmentally damaging footprint is inconsistent with the environmental stance most US public companies. The U.S. government focuses on carbon footprint reduction and in the courtroom of public opinion.
Private chains such as Hyperledger FabricThey rely on 1990s era peak capacity approaches that do no support auto-scaling, or other dynamic capacity mechanism. While they are more efficient that Ethereums proof of work protocol, they suffer massive under-utilization in data storage mechanisms. Their need for heavy, constantly on compute capacity drains power and leaves a carbon footprint 24x7x365, regardless the actual transaction rates.
Moderner approaches include Vendias blockchainInvest in more efficient systems Serverless technologies Cloud services that are sustainable for the public. Modern blockchains, which use cloud-native technology, offer tight cost enveloping with a carbon footprint that is actually quite low. LowerHosted databases and APIs are more efficient than traditional (centralized) IT methods for sharing data. IT teams can use features to reduce file redundancy to increase storage efficiency and security without compromising functionality. Companies and enterprises of all sizes can benefit both from the speed of delivery as well as the improved cost and carbon footprint outcomes of SaaS-delivered Blockchain capability using these newer methods. This allows them to create cost-effective cross-cloud data fabric, partner and operational data service solutions, while simultaneously OptimisingTheir carbon footprint stance.
First-generation chain: Promising tech, unacceptable environment costs
Climate change is a constant topic of media attention. This attention is shared by government and private sector efforts to reduce greenhouse gas emissions. The current U.S. administration has taken several steps to reduce carbon footprints and the resulting environmental damage. These include programs that target supply chains, power production, as well as SEC reporting requirements for public corporations. CIOs have been thinking about lowering greenhouse gas emissions and improving IT efficiency for some time. However, this increased transparency is only the beginning of a push to ensure compliance that will eventually rival SOC or PCI in terms of its impact on R&D and business operations as well as investor reporting. Companies, especially larger ones, must start planning. Nowfor the inevitable effect of exposing their IT portfolio choices the broader public.
Blockchain technologies provide companies with a promising platform for building everything. This includes operational data store (ODS), systems that can span public clouds providers, and partner data sharing. These solutions replace traditional API-based solutions by blockchain-powered smart APIs. However, the environment is not well-suited for first-generation blockchain technology.
- Ethereum, once referred to as the world’s computer, shares with Bitcoin a destructive proof-of work algorithm that is actually designed for large amounts of computing power to disincent fraudulent transactions. This particular approach has had the effect of blocking mining and causing countries with low electricity costs to shift to them. These countries are often based on dirty production methods like coal mining, which exacerbates pollution and increases carbon footprint. Bitcoin, for instance, already consumes more electricity annually than many entire countries. Expanding Ethereum’s use by 10-20 orders, as would be necessary to give it the processing capabilities of conventional IT operational systems, would have an untold impact upon the environment. Although the Ethereum community has been discussing the possibility of moving to more efficient mechanisms for a long time, there has been little real-world progress over the past several years.
- Private and permissioned chains (a category dominated in Hyperledger Fabric) continue to rely on last century scale to peak capacity approaches. Hyperledger Fabric and Quorum are not cloud native systems that can exploit more efficient container packing or serverless technologies. Instead, they rely on single-server deployments that do not offer any internal scaling mechanisms. This makes it difficult to apply autoscaling or other capacity management methods. This creates an always-on solution that utilizes 100% computing, database and storage capacity, even when no work is being done.
Public opinion has associated blockchain technology with a high and unacceptable carbon footprint. If used correctly, blockchains can actually reduce carbon footprint. Modern blockchain protocols focus not only on increasing cost effectiveness and ease-of-use, but also on improving compute and storage efficiency. This allows for a decrease in carbon emissions relative to traditional IT approaches.
In cryptocurrencies and other public blockchains proof of stake has largely replaced the proof of work in modern implementations. Although proof of stake is sometimes criticized for being another form of centralization it doesn’t leave the same carbon footprint as the Sybil attack-resistance Proof-of-work approach. Public chains serve a large global ecosystem, so at most the most popular ones enjoy a reasonable amount of usage.
Public chains still suffer from inefficiency. Even when using proof of stake, they are required not to use their computational resources to compute actual results but to maintain Byzantine or denial-of service attack resistance. They must also maintain a least-common-denominator approach to data storage and modeling that can be used by anyone in their community. Optimizations based on access patterns or data models cannot be relied upon.
Worse, because public chains are public by design, every node must keep a copy and update all information from all sources. Every node within the network must have access to all data and updates, including experimental or testing data from a no longer-existent startup. If two companies want to use the public chain to communicate but don’t necessarily want (or need) others to join, then every other node (and any auditing clients listening for updates), still needs to be informed. This makes data distribution easier. Data storage is extremely inefficient over time because of what the intentionally access pattern neutral approach to public chain design. These problems can be solved by techniques like Sharding L2 cachesEach approach has its own problems, such as being more centralized and putting the burden of choosing a subcommunity to communicate on every client.
These public chain drawbacks don’t improve with time or technology. In fact, they get worse as streamed data increases and total data stored grows. Worse. Private chains will continue to be a more efficient and sustainable technology for applications such a partner data sharing, cross cloud operational data stores, real-time data fabrics, and other data storage than public chains.
First generation private chains like Hyperledger Fabric or Quorum rely on known identities of node operators. They don’t require Proof of Work nor Proof of Stake to safely mint blocks. As far as data sharing and storage platforms go, they are much less efficient than modern cloud-based approaches to storing, sharing, and transferring data such as Amazon DynamoDB and Azure CosmosDB. These cloud-based solutions are more efficient than traditional infrastructure and electricity for many reasons.
- They can achieve an aggregate utilization that is higher than any individual company or deployment through sharing of resources without having the need to compromise burst capability.
- Their storage capacity is constantly expanding, avoiding scale-to-peak concerns that can lead to over-provisioning legacy blockchain storage resources.
- They can adjust their fleet size and work allocation algorithms to ensure that they direct compute power to where it is needed. This helps to avoid scale to peak concerns which can lead over-provisioning legacy blockchain compute resources.
- They are inherently fault-tolerant across containers and servers as well as available (faulty) zones. This eliminates the need for applications to create redundant deployments. Legacy blockchains, on the other hand, require multiple nodes to overcome server and availability zone failures. This results in a larger computational footprint and storage footprint to achieve the exact same result.
These challenges have been solved by public cloud services. centralizedData sharing solutions make it easy to wonder if other data sharing solutions could not be used. DecentralizedData sharing solutions, i.e. blockchains. Indeed, second-generation blockchain approaches have done exactly that.
The greenest chain possible: Serverless blockchains
Many public cloud services are now considered serverless. Although the term may seem ironic given their obvious dependence on servers, it conveys important elements of both developer experience, and implementation efficiency.
- Multi-tenant massiveServerless implementations take multi-tenant cloud computing to the extreme. They allow a smaller number of shared resources than individual servers to process workloads for different customers without compromising security and operational workload isolation.
- Scaling per-requestServerless methods generally integrate automatic scaling directly into their algorithms. Every request to them is also implicitly an instruction to the platform as the whole. This allows the platform to recruit more (or less) resources and allocate workloads across the vast fleet intelligently. This feat is impossible for any company or limited on-premise deployments. A serverless approach, unlike traditional blockchain approaches, can automatically scale to meet requests. This allows them to be cost-effective and efficient per transaction.
- Implicit fault toleranceA massive fleet and dynamic workload allocation (re) offer the advantage of protecting each customer’s workload from outages on individual machines or entire availability zones. This is possible without having to modify or write code. Blockchains built on serverless technologies are more fault-tolerant than traditional blockchains that are based only on single-server deployment models.
- Scales up to zeroServerless applications are able to turn off completely, unlike traditional deployments such as those used in the first generation blockchains. This means that they do not have an ongoing footprint and can be shut down when no work is being done. Instead, other customers of the services use those (multi-tenanted), infrastructure, HVAC, electricity and other resources to gain useful work. Importantly, eliminating this baseline expense also eliminates the equivalent carbon footprint of the customer.
These many advantages of serverless technology are passed on to platforms that are built from themThis is also true for serverless blockchain technologies like Vendias. They not only enhance on existing private blockchain technologies that are always online, They are actually better than the more traditional (centralized) methods of building data sharing platformsAs the next section explains,
The utilization of commercial IT servers and conventional data centers is not uncommonly lowEstimates range from 5-15% (i.e. 85-95% waste). This is not surprising as every company’s solutions and applications have their own usage patterns. It is impossible for any company to build their own version of a serverless computing platform without sourcing third-party workloads. This is a problem that even the Fortune 50 has. Everyone else’s isolated workloads will result in low server utilization rates even if they are running on the public cloud.
Companies that want to share data between departments or organizations, or with business partners (in supply chain arrangements or multi-company arrangements), will find themselves in a difficult position. They must build custom implementations to host the APIs and connect them to the data. These implementations are complex distributed systems that require high-caliber engineering talent to create and deploy. They also require 247 operations support. They allow data to travel between companies, clouds, or organizations that have different compliance regimes. This makes them vulnerable to high levels of scrutiny and risk in terms of security, regulation enforcement, and policy enforcement. They are also single-use applications and therefore have low usage. CIOs and CEOs will have to manage a large portfolio with poorly used IT solutions.
Modern blockchains offer a unique solution. They make it easy to share operational data with partners, reduce time to market, eliminate project and security risk, and minimize the undifferentiated work of creating redundant code and data-sharing platforms. Companies can simultaneously reduce their 10% utilization of homegrown solutions to 100% by using modern, serverless Blockchains. Serverless solutions are only active when the actual work is being completed, which is usually during construction. These blockchains can be delivered as a SaaS service, which allows companies to dramatically reduce the staffing needed to develop and maintain the systems. This shifts much of the work onto the public cloud and the blockchain service providers, helping to lower IT costs. The multi-tenanted nature and security of the cloud infrastructure allows companies to benefit from the safety and security of professionally managed fleets and software systems, which are outsourced and staffed around the clock. Serverless blockchains can be used to increase utilization, reduce environmental impact, speed up time to market, and lower cost than traditional methods of building data-sharing solutions like public APIs.
Efficient file sharing solutions
Although databases are the star of enterprise data storage and sharing, the majority of data owned by companies is actually in form of Files. How files are shared, stored and exchanged, duplicated, and managed has a greater effect on greenhouse gas emissions that database storage. Files are crucial to partner data sharing solutions as they often serve as the basis for both industry and de jure data exchange standards.
Modern blockchains handle files on chain as well as scalar (database held) data. They treat them like native data types. It is important to avoid duplicating large amounts of (often large!) data files. Also, it is important to not create redundant copies of the data in any partner’s IT stack.
Vendia’s blockchains include sharing controls and dynamic files exchange to achieve this. Customers can choose from fully redundant copies (maximum operation isolation but also the lowest environmental impact from redundant storage) or fully dynamic, which only stores one copy and is fetched when requested by other users with the appropriate permission. Hybrid strategies include caching (fetch the first time) and quorum, which allows for a limited number of copies to be kept in strategic locations (e.g. one per public cloud). These crucial operational controls, along conventional governance access controls, would allow redundant file storage to rapidly balloon out of control, negating any gains from improved sharing of scale data. This is why many public-chain file sharing solutions like IPFS FileCoinThey have not been able to reach even a small percentage of cloud data storage solutions like Amazon S3. The high cost, high latency and low throughput of such systems makes them unsuitable for sharing most IT file sharing needs such as partner data exchange.
Blockchain technology can be disruptive (Bitcoin and Ethereum), or merely low-use (Hyperledger Fabric, Qorum), but IT professionals must be careful to choose technology that is both cost-effective and will maximize application utilization. Companies should present their businesses in the best light possible when carbon footprint reporting becomes fully effective. The following list will help you identify technologies which can improve a company’s carbon footprint rather than causing it to decline.
- Does the solution use proof-of work algorithms?Even if used in an indirect way, proof of work can be a source of very high carbon emissions and dirty fuel consumption that could damage a company’s public image.
- Does the solution allow for on-chain file storage with high availability and redundancy as well as low latency and IT ready redundancy controls?Files make up a large part of an IT organization’s storage footprint. Without a solution to manage and track content duplication and exchange, redundant files will quickly overtake efforts to share data effectively. Redundancy and availability should be built-in features. Not client- or app-derived outcomes.
- Is the solution serverless? Or is it a single server?Serverless solutions can be used to build 100% applications, and also provide built-in scaling or fault tolerance. Single machine deployments can’t scale and aren’t fault-tolerant. This can negatively impact a company’s carbon footprint.
- Is the solution available as a SaaS?SaaS services not only reduce development and maintenance costs but also allow multi-tenanted approaches which increase efficiency, lower aggregate costs, and reduce carbon footprint.
In just a few years, the issue of protecting the environment has moved from being a fringe concern to a top concern of nations. This has had a significant impact on domestic and international policy. With the new reporting requirements and increased compliance from corporate reporting likely, it is now that CEOs and CIOs need to evaluate their IT choices. This will allow them to develop strategies to reduce carbon emissions over time. Companies will be able to identify areas for improvement by focusing their attention on compute and data, which are the key drivers of power and cost. It is important for IT decision-makers and architects at all levels to understand and identify which blockchain technology and providers can help reduce carbon footprint. The checklist can be used to help companies make informed vendor selections and guide them towards a carbon-efficient and cost-effective solution.
Tim Wagner is the co-founder and inventor of AWS Lambda. He also served as the general manager of AWS Lambda’s API Gateway services. He also served as Vice President of Engineering at Coinbase.
VentureBeat is a community for you!
DataDecisionMakers allows experts, including those who do data work, to share their data-related insights and innovating.
DataDecisionMakers offers cutting-edge ideas, the most current information, best practices and the future data and tech. Join us to learn more.
You might even consider contributing an article of your own!
DataDecisionMakers – More