I Know It Was MEAN, But I Cut the Cord to LAMP Anyway
•Download as PPTX, PDF•
0 likes•3,989 views
All Things Open 2014 - Day 2 Thursday, October 23rd, 2014 Brian Hyder Co-Founder & CTO of PencilBlue, LLC Back Dev I Know It Was MEAN, But I Cut the Cord to LAMP Anyway
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to I Know It Was MEAN, But I Cut the Cord to LAMP Anyway
Similar to I Know It Was MEAN, But I Cut the Cord to LAMP Anyway (20)
More from All Things Open
More from All Things Open (20)
Recently uploaded
Recently uploaded (20)
I Know It Was MEAN, But I Cut the Cord to LAMP Anyway
- 1. I Know it was MEAN but I Cut the Cord to LAMP Anyway
- 2. The Next Milestone in Web Development info@pencilblue.org @GetPencilBlue
- 3. Planning Development Deployment Actually do it Managing the flexibility What are my options? Do I need a framework? The Basics The Upgrade Dealing with SEO Simplifying Client Code Server Options Finding the Right Talent Promise vs. Callback Choosing a Cloud
- 5. MongoDB
- 6. Planning - Failing to Plan Pros Cons Easy Install Not good for inter object relationships Flexible Quick Development
- 7. Development - Prototyping vs. Production
- 8. Development - Flexible Base Service + save + delete + load Widget Service + save + delete + load Widget Service + save Sprocket Service + save { object_type: “widget”, name: “”, description: “” }:
- 9. Development - Too Flexible? Document with Strings { object_type: “event”, type: “startup”, location: { lat: “35.7806”, lon: “78.6389” } } Document with Floats { object_type: “event”, type: “startup”, location: { lat: 35.7806, lon: 78.6389 } } ● Valid JSON ● Pases non-strict validation (JS) ● Geospatial index fails on insert
- 10. Development - Schema Documentation
- 11. Development - Common Data Fields { “first_name”: “Charlie”, “last_name”: “Daniels”, ... “doc_type”: “user”, “doc_version”: 1, “created”: “2014-09-11 15:01:07.602Z”, “last_modified”: “2014-09-11 15:01:07.602Z”, “created_by”: "5411b9334572668410000fd0”, “last_modified_by”: "5411b9334572668410000fd0” }
- 12. Development - Geospatial Indexing The $geoWithin operator does not return sorted results. As a result MongoDB can return $geoWithin queries more quickly than geospatial $near or $nearSphere queries, which sort results. db.places.find( { loc : { $near : [50,50] , $maxDistance : 5 } } ); db.places.find( { "loc": { "$geoWithin": { "$center": [[50, 50, 5] } } } ); ==
- 13. Deployment - Expense at Scale Deployment Cost Managed AWS m1.large $0.20/hr NO Mongo HQ $18/GB* YES Mongo Lab $15/GB* YES Linode Medium $20/mo NO * Other base fees may apply
- 14. Deployment - MMS & Getting to Scale
- 16. Planning - Do I need a framework? ● IS IT OPEN SOURCE? ● How often will I need to upgrade? ● If the project goes stagnant does that affect me? ● If the project gets bought by a company I don’t like does that affect me? ● What kind of custom functionality do I need to inject? ● Will packages be sufficient or will I need to fork so I can experiment? ● Do I want to maintain something custom? ● Can I do as well as or better than what is already out there?
- 17. Planning - Rube Goldberg of Software
- 18. Planning - The Forum Conundrum
- 19. Development - The Basic Example var express = require('express') var app = express() app.get('/', function (req, res) { res.send('Hello World!') }) var server = app.listen(3000, function () { var host = server.address().address var port = server.address().port console.log('Example app listening at http://%s:%s', host, port) })
- 20. Deployment - Upgrading What happens when I try to upgrade? Chance of Conflict Time
- 21. Angular
- 22. Planning - Google did it!
- 23. Planning - Weighing In
- 24. Planning - Crawlers and Javascript Pros Cons Server Rendered Landing Pages SEO becomes easier because it is the traditional model. Even mediocre crawlers can navigate your site Mix and match of SPA and traditional model Tracking which pages need to be rendered server-side Single Page App Consistent SEO implementation Metadata data changes based on page context Google is only search engine boasting javascript execution.
- 25. Development - A Simple Example <ol> <li ng-repeat="user in users" ng-bind="user"></li> </ol> //service code angular.module('myservices', []) .service('simpleService', function(){ this.getUsers = function() { return ['John', 'James', 'Jake']; } }); //controller code angular.module('myapp', ['myservices']) .controller('SimpleController', function($scope, simpleService) { $scope.users = simpleService.getUsers(); });
- 26. Development - A Growing Stack Data Access Service Layer and Business Logic API Controller Angular Service Angular Controller Client Server
- 27. Separation of Concern => Testability describe("Unit Testing Examples", function() { beforeEach(angular.mock.module('App')); it('should have a LoginCtrl controller', function() { expect(App.LoginCtrl).toBeDefined(); }); it('should have a working LoginService service', inject(['LoginService', function(LoginService) { expect(LoginService.isValidEmail).not.to.equal(null); }]) ); });
- 28. Deployment - Serving a Single Page App? Considerations: ● How much data do I need to persist? ● Is my application solely a SPA? Possible Options: ● Apache / PHP ● Python ● Rails ● Node.js
- 29. Node.js
- 30. Planning - Resource Management Now that I use the same language across the entire stack hiring just got simpler.
- 31. Planning - Resource Management
- 32. Planning - Adoption Big names are jumping on board: ● Linkedin ● Walmart Labs ● PencilBlue http://nodejs.org/industry/
- 33. Development - Node Inspector
- 34. Development - Clustering if (cluster.isMaster) System.onMasterRunning(); else onChildRunning(); System.onMasterRunning = function() { //spawn workers var workerCnt = os.cpus().length; for (var i = 0; i < workerCnt; i++) { cluster.fork(); } cluster.on('disconnect', System.onWorkerDisconntect); pb.log.info('System[%s]: %d workers spawned. Listening for disconnects.', System.getWorkerId(), workerCnt); }; Core 1 Core 2 Core 3 Core 4 Core 5 Core 6 Core 7 M W W W W W W W
- 35. Deployment - Shooting for the Cloud Elastic Beanstalk OpenShift Nodejitsu Bluemix Linode Heroku
- 36. Deployment - Starting Simple L B M W M W W W Use SSL Termination Pay for the backup service Write a procedure for creating these nodes
- 37. Pros Cons Extremely Flexible Rapid Development Expensive at Scale Fast “Good Enough” HTTP Server Upgrades cause code changes A battle tank that can do anything Testable Overhead for smaller projects Fast and efficient Hard to debug To Recap...
- 38. The choice is yours but...
Editor's Notes
- My name is Brian Hyder and I am the Co-Founder and CTO of PencilBlue, a startup here in the Triangle. PencilBlue is a CMS built for the cloud and provides all of the tools you need for the way we use the web today. The entire platform is built on the MEAN stack and we ran into interesting challenges along the way. A startup’s resources are limited and you want to use them wisely. Choosing the right technologies can be a significant competitive advantage so I’m here to share our experiences with the MEAN stack and to help empower you and your team to find the technologies that are right for your project.
- Today’s talk will cover the pros and cons of the MEAN stack, starting with MongoDB. We’ll cover each of the MEAN components pointing out some of the gotchas we have encountered along the way. Each component will be broken down into three sections: planning, development, and deployment.
- The presentation is available at: https://pencilblue.org/presentation
- The first component we will talk about is MongoDB. It serves as the foundation of the MEAN stack acting as the data store. Initially released in 2009 by 10gen. What is now known as MongoDB, Inc. The name was changed to move the company into alignment with its flagship product. Document storage based providing support for multiple data types BSON format (Binary JSON).
- The database is the foundation of whatever you build. The strengths of the DB should align with the most complex problems that you and your team will need to solve. When we started PencilBlue we didn’t do any research to see which databases were best suited for our particular problem. We knew we wanted something easy to install, flexible data structures, and quick development. MongoDB solved all of these problems for us with ease. We started development and never looked back. It wasn’t until we began to address the complexities of the relationships among our system objects and scale that we realized that some of our challenges didn’t fit the strengths of the chosen database. The complexities of inter object relationships is not what NoSQL was built for and is something we should accounted for during our research phase. Your team should take the time to fully understand the complexities of the project and decide which DB to use because once you make your decision its hard to stop development.
- Our experience has been that Mongo has been the easiest to get data flowing from the DB to the UI and vice versa when using the MEAN stack. The driver is available nia NPM and there are a lot of rudimentary examples on how to get started. However, we have also seen where over time complexity grows to the point where a relational database would have been more practical based on the relationships among objects even with the flexibility provided by nested documents. This isn’t Mongo’s fault but a mistake on our part. We overlooked the complexities of the system we were building.
- The architecture of PencilBlue is a pretty typical. It is a layered approach where there is a data access, services, and controller layers. The approach we took was that each system object gets its own service prototype that wrappers a portion of the data access layer. We found that by doing this we had a large number of lines of code, primarily CRUD operations, were the same. The flexibility of working with document storage allowed us to easily abstract common code into a base service prototype that could be inherited from to provide custom operations such as validation. The flexibility of document storage provided fast and efficient development to execute object interactions. However, there was one flexibility issue we overlooked.
- In one particular case we were attempting to track events triggered by a native mobile application. The event object sent from the mobile device to the server contained various device information along with a value for latitude and a value for longitude. We were collecting data for awhile before we decided to build a UI around the data by interacting with the Google Maps API. In order to do that we needed to place a geospatial index on the lat/long fields. What we didn’t realize was that one platform was sending this data as strings and one was sending it as floats. When we attempted to create the index it failed due to the inconsistency of the data types. Our validation routine was not validating the type just that we were getting a numeric value. The lesson learned is that even though you are afforded the flexibility stick with specifically typed data as best as you can. This enforcement would have been provided for free in an environment where the schema is decided ahead of time. The second layer of validation in the form of type checking would have protected us from ourselves.
- The flexibility of Mongo allows users to build out documents in a collection anyway you wish. The only time when Mongo cares is when index criteria are not met. You can’t and shouldn’t index everything so think about your schema and white board it out. This fact caused an interesting side effect in our development process that hadn’t really been prevalent before. We began creating wiki pages to explain the system objects and their interactions. Since there is no schema it is up to you and your team to generate the documentation. Build time into the development cycle to do this. We found that early on our data models were constantly changing and team needed to be notified when these changes occurred. It also helped our QA team know how to craft automated tests in order to match the new data models. Wikis come with almost any git hosting provider. Use them and save yourself and any newcomers time by pointing them to accurate data.
- We found over time that each system object benefited from having a consistent number of fields. It helped from an auditing standpoint but also from a data migrations perspective. Our data model was constantly changing in the beginning. Although the product was new there was already data rolling in and so everytime the data changed that meant that we either needed to write a migration as the data changed or update the code to be backward compatible. We decided that data migrations would keep the code streamlined but we would need a way to track which objects were associated with what version of the object schema. We decided add a version field to each document to allow us to select only those objects that needed to be migrated when doing conversions. These common fields also meant that we had a common set of indexes and filters across the system. We decided on the following set of fields: doc_type: The type of document within the collection. 99% of the time this should match the collection name doc_version: The object’s schema version. This number is object independent or you can match it up with your tags. created: When the object was created last_modified: When the object was last persisted created_by: The user or entity that triggered the creation of the object last_modified_by: The user or entity that triggered the persistence of the object
- Sometimes you have to try something out against production. We didn’t have enough data in our local environments and we were too resource constrained for a test environment. Either way it was still a terrible idea and we learned a very hard lesson one day. Once we ensured that we had consistent latitude and longitude data in our events collection we were able to create the geospatial index. We read the documentation and built out a query using the $near operator. After all, the goal was to find all events that occurred within a given time frame and within so many miles of a specified coordinate. The collection was indexed appropriately as a 2d sphere but we crashed the DB when using the $near operator. The mongod processed used all of the available memory on a medium AWS instance and never freed it back up. A process restart was the only method that would clear out the query. After almost scrapping the whole thing and going to Postgres we did a little more digging into the documentation and found this gold nugget tucked away on the documentation for another operator. The $geoWithin operator was what we were actually looking for and had vastly greater performance. When it comes to building out index research all available options and the operators that will be executed for each query against it. MongoDB has provide a vast amount of documentation it is up to the developers to read it first before executing against their deployments.
- One project I was associated with we were tracking events and at first we were seeing them trickle in, then we saw a rate of 1/s and were ecstatic to have the data. As the product expanded to more apps we found that the rate increased to over 22/s. This added up quickly and we realized we would need to shard early. We already ran a 2 node replication configuration (master/slave) but needed a full blown replica set to happen soon. We examined many hosted solutions and even managing it ourselves in AWS. The AWS recommendation was to use a m1.large with 1000 provisioned IOPS ($0.20/hr or 144/mo). Managed hosting options are available including elastic deployments. The two front runners include Mongo HQ ($18/GB) and Mongo Lab ($15/GB). Be mindful of these solutions as they are usually multi-tenant VMs and restrict how clusters are sharded. Although I’ve never used tinkered with either one I’ve heard that other fees apply to these solutions. In the end the sheer cost in addition to the complexities of the data forced a move to Postgres. Cost was the predominant factor in our decision which is not uncommon for a startup. If you are lucky enough to know that your collections will stay relatively small or you have plenty of time before growth then I would recommend at managing a small cluster yourself. Look at simple hosting providers such as Linode. This was the direction we took at PencilBlue. Our initial site ran on a medium Linode instance and and we never had an issue. We ensured that backups were taken daily and that satisfied our needs. You and your teams should try out a couple of different options. Hosted and self-managed to get a feel for what your team can handle based on budget of time an money.
- Traffic to PencilBlue’s site is relatively predictable however it is not uncommon to get a spike in traffic after we promote a blog post. These are the days that I have Google Analytics running in a browser window all day. We needed a monitoring solution to know when the DB was getting hammered. We didn’t have the time to setup Nagios or Ganglia and a paid solution was not going to provide enough benefit to justify the cost. We turned to the Mongo Monitoring Service (MMS). It was a brilliant move on the part of Mongo was to offer free monitoring with MMS. The pain of monitoring your database is lifted; FOR FREE. We’ve seen instances where client support was notifying us about an issue before we knew about it. The only challenge with MMS is that it features two factor authentication. One project I worked on we worked out of a basement where wifi and cell reception were almost non-existent. Authentication codes would expire by the time the text message arrived. The alternative is their integration with Google Authenticator but you will need wifi to generate the codes. In a recent announcement MMS is offering a deployment mechanism in the newest version. Not all of the kinks have been worked out yet as noted in the announcement but they appear to be working on it. For instance, you cannot upgrade an existing deployment to the new deployment service. With deployment taken care of your teams can now focus on the things that really matter, your code and frameworks.
- For those unfamiliar with Express it is a routing framework now maintained by StrongLoop. It is responsible for creating a HTTP server within your Node.js application. It allows developers to specify route definitions and pair it with a function to be executed when an incoming connection matches the route. koa, appears to the be the next generation of Express.
- I’ve worked with projects that needed an underlying third party framework and others that went completely custom. An example of a project that needed a framework was a project that focused more on a functional API with typical middleware (passport, body-parser). Express was the low level plumbing that was “good enough” to focus on the pieces that mattered. The speed was acceptable because the middleware was kept to a minimum. The API supported a UI as well as calls from native mobile applications. The API served millions of requests per month. An example of a project that didn’t rely upon a framework was PencilBlue. It is a CMS that provides the ability to prioritize the controllers that were executed for a route based on installed themes. This functionality proved to be difficult with Express 3. In addition, we had concerns about drastic changes that Express would make and/or the potential for their direction to change based on an acquisition. These assumptions turned out to be correct and by providing a custom solution we were able to streamline a custom framework to suit our needs. You and your team need to decide what balance of complexities you are OK with. Choose a framework and experiment with with. Build out a prototype and see if it covers what you need it to do. If you do decide on one or more frameworks take the time to ensure that they play well together.
- I was working on a project where we knew that we were going to use express. We also knew that we would also need to support multiple forms of authentication. The natural choice was passport, a package that provides base authentication models and is extendable for other patterns. In the beginning we kept playing with different packages to see which ones would play nicely with the version of Express we had gone with. It turned out to be somewhat troublesome for us despite the fact that we only needed to support API key and basic authentication. With so many frameworks and middleware it is easy to get bogged down in doing more searching for packages to make your life easier and less designing a product that actually accomplishes your goals. Don’t make the mistake of spending all of your time trying to wire it all together. Sometimes you just have to write it yourself. Even Stack Overflow can’t help you.
- We spent a lot of time studying how to plugin all the middleware. What this really means is that I spent a lot of time searching forums for the right answer. What I found was that just because I saw the green checkmark on Stack Overflow from 2012 doesn’t mean it still applies. I would see the green check mark of hope followed by the answered date of two years ago followed by recent statements of, but this is still the old way of doing it. While Node.js and NPM do a fantastic job at solving dependency issues some of the frameworks and their middleware do not. In addition, the infancy of a large portion of the frameworks out there cause major changes to be introduced more frequently. For example, Express has had 213 releases since its first release on January 3rd, 2010. That’s a release every 8-9 days. The one thing almost always missing from a forum post about how to do something is the version of the tools they were using. When selecting your framework, do so heavily on the quality of THEIR documentation and get comfortable with the source code. You might find that you spend more time with it than you thought in order to get started.
- Express has made it extremely easy to get started and start building out actual routes. Although this common example isn’t exactly how you would see it laid out in a production environment it does serve to show how it could be done by creating a repeatable pattern of a single function to handle every route. What we have done in the past that has worked for us is to build a small routine that loads all system routes by iterating over a descriptor that points to where the handler function is and any security constraints around that route. The pattern shown here for Express has essentially been the same for some time now but that doesn’t mean it won’t change.
- Frameworks usually work out to your benefit until you hit the one use case that has an open bug in it. At this point you have two choices: fix the bug and submit a pull request or wait for a release with a fix. Either way at some point you must still perform an upgrade on the framework. Software upgrade of any kind come with inherent risks that increase over time. We saw some of this pain during the transition of Express 3 to Express 4. Prototypes were built in version 3 but with the move to production we wanted a few of the features that came along with version 4. This caused a day or so of conversion work for us. The whole problem is the lack of maturity of Node.js as well as the frameworks that are built on top of it. Each change at the Node.js level can cause a ripple effect that forces changes all the way up to stack to your code. The longer you wait to upgrade the greater the chance that things have changed so drastically that migrating your code will be difficult. You and your team should decide upfront if the version is good enough or if there is something that you need that is in an upcoming release.
- Angular is a client side MVC framework that derives its power from the combination of CSS and javascript. It was originally developed in 2009 as proprietary software. It was finally released open source 2010.
- When it came time to implement the front end of PencilBlue we knew we had choices. We could go in a million different directions. Everything from React, Backbone, or just straight up jQuery. We decided on Angular. The biggest plus for us was that it was developed by Google which turned out to be false. It was actually designed by Misko Hevery (known as the Father of Angular on Twitter). However, Google is now overseeing its development which for us means a decent lifespan and large community support. Essentially, it provided everything we needed from a development standpoint and more. The “more” piece of that statement is important because PencilBlue needed to be able to bend to the will of the developers.
- The best analogy I’ve heard is that Angular is a battle tank. It can do anything you need it to. This includes bringing order to the chaotic world of client side javascript. In PencilBlue’s case it was clear something like Angular was needed but it might not always be the right answer. Why use a tank when all you need is a basic sedan? Larger projects will most likely benefit from the structure and flexibility that Angular provides. Take the time to do a POC with Angular and with some simpler framework such as plain old jQuery or React. It just might save your client (or yourself) time and money. Use sites like BuiltWith.com to check out competitor sites and see what they are built with.
- As a startup it is incredibly important to get your name out there as fast as possible. The best way to do this is by optimizing your site for SEO. As part of our core platform we included all the necessary tools to populate all of the meta-data tags for each page and built in a site map but there was still one issue we couldn’t get around. Angular apps are dependent on the execution of javascript to render properly and most search engines just don’t do it. The amount of time taken to consume all dependencies and then execute just to render is vastly greater than that of a traditional static web page. However, life got simpler on May 23, 2014 when Google announced that their crawlers were executing javascript in an attempt to crawl a web page as a user sees it. This means that you don’t have to plan on the public pieces of your site to be traditional server-side rendered HTML. The cost of that can be transferred to the browser. Translation - lower cost in processing time and potentially, bandwidth for content producers. Be aware that Google said it themselves that the execution is not without flaws. Developers will still have to decide if it is important to support all crawlers or just Google if they go with the fully Angular app. In the case of PencilBlue, we built the platform to support a hybrid model. Developers can choose which pages are rendered with Angular and which ones are rendered server-side. This approach has been successful in helping us to rise through the ranks of Node.js CMS search results.
- An oversimplified example for the unfamiliar. The code takes a small snippet of HTML to create an ordered list of users. The javascript executing behind the scenes has retrieved the user list and that list is now watched. Any change will be reflected in the list displayed in the browser. The ng-repeat (my-favorite) is what makes this possible. It provides a construct for iterating over arrays.
- Our experience with Angular has shown that it forces developers to take a cleaner approach to client side javascript. Essentially, less chaos and more structure. In order to accomplish this Angular creates layers: a module, 1 or more services, and 1 or more controllers. The drawback is that it does create another layer and potentially more code. Across different projects this varies. For simple projects it appears to be overhead whereas larger project benefit from the lack of duplicate code that would normally be spread out across javascript for specific pages. The layers were always there they just might have been merged. When working with Angular follow established patterns. It will create cleaner maintainable code and make it testable.
- The separation of concern that Angular encourages also makes your code testable. In the world of Single Page Applications the UI is almost completely segregated from the the server side code. This means that the code base for the UI should be tested independently and together. Jasmine is a great tool to make this possible. It semantically provides a framework to test Angular along side of Angular’s built in testing features. This simple test sets up a test suite that mocks out the module and has two test cases. The first ensures that the module has a login controller while the second ensures that the login service gets successfully injected into the app. Teams should take the time to write the same unit tests that they would for the backend layers. The testing features that Angular bring to the table empower the UI engineers to enhance the quality of the UI so take advantage of it.
- Determining how to serve a SPA comes down to two considerations. How much data do I need independently of the API at bootstrap and will I have traditional server-side rendered views. The first is typically a product of security, i.e. storing authorization codes and such. In the case of a hybrid model you will need a server that has the ability to render more than static pages. This makes the deployment more complicated. When you need something simple Apache is going to be a solid choice. if you need something quick and out the door whether it is static content or a few simple templates put together the Apache/PHP approach might be best. More complex scenarios where you need contextualized server-side rendering you might find yourself wanting to align yourself with the technology you chose to build your API. This is because you will most likely be leveraging the same service code to access data. Using the same code that is already in place for the API will keep the code base tidy without duplicating the effort.
- Node.js is server side javascript that runs on Google’s V8 Engine. It was created in 2009 by Ryan Dahl at Joyent.
- People claim that a large advantage to Node.js is that everybody now speaks the same language across the entire stack and that a side effect is that the hiring pool just got larger.
- We were looking to backfill a position for a full stack developer. We obviously wanted to somebody who knew Node.js well and we thought because Node.js was javascript we wouldn’t have any issues finding a good candidate. We were wrong. The majority of time the knowledge of the javascript language wasn’t the issue. We found that the mindset of front end developers tended to be different from that of the back end developer. The front end developers were focused on the user experience and had a great eye for aesthetics but sometimes faltered on the infrastructure and how to scale. On the flip side the back end developers knew about architecture but only to API layer and thinking asynchronously isn’t something that they have traditionally had to do. It explains perfectly why my drawing expertise ends at stick figures and I struggled with promises. We also found a few instances where people list multiple years of Node.js experience. While this is possible it isn’t as plausible. Always double check to make sure that their javascript experience doesn’t blur with Node.js experience. They can be vastly different. We finally found an excellent candidate. He had experience with Node.js and some Angular. More importantly, he was willing to learn and was passionate about what we were doing. In the end, that is what really mattered. Already being familiar with the technologies is an advantage but not required. The candidate needs to be willing to learn and embrace a challenge. However, if you have a hard time finding the right people then why bother to use Node.js?
- A lot of developers are wary of new technologies. Why should we change when there are technologies that have been fully battle tested? The reason is because these new technologies are built to save you time and money. This is why large players are using it. Walmart Labs is a contributor to the Node.js code base and LinkedIn and Uber use it for their mobile APIs. We can also look at the adoption across the rest of the developer community. The number of new javascript repositories are increasing while other languages are decreasing. What this tells us is that people are being successful with Node.js despite its infancy and it is worth a try.
- When we started out learning Node.js and building PencilBlue we were constantly writing code that had bugs in it. Not surprisingly enough, nothing has changed about that. What has changed is that we realized that there was a tool available to help us go from “console.log” statements everywhere to a UI to allow us to step through our code, line by line. The project is called Node Inspector. It is a project sponsored by StrongLoop and has done a great job at providing a full UI around the Node.js debugger. There is one critical flaw with the debugger. If you use the cluster module then it doesn’t work. Each child process attempts to use the same port for communication and causes the initial process to disconnect from the debug UI. The issue is being fixed in Node.js v11. You and your team should get familiar with the debugging tools. Our experience has shown that they have a bit of a learning curve but do provide value. Mostly, in terms of time.
- A CMS needs to scale easily. One of our primary focuses was to ensure that PencilBlue would horizontally scale. Node.js’s design encourages stateless web server design. In conjuntion it forces developers to deal with scale right out of the gate. In order to do that we had to leverage the “cluster” module that is part of the core Node.js API. Clustering allows you to spawn child instances of your application and have each process attach to the same port to listen for incoming connections. The example shown here illustrates how a Node.js application can scale out across all cores of a VM or physical server. The first process started becomes the master. It is responsible for spawning off the first set of children. After it has successfully spawned all necessary children it begins to listen for children that have died off. The master process then can decide whether to terminate all children or spawn a new child. Although the master process will spawn a new child it is still important to deal with uncaught errors. -----Extra-Info-Not-Presented------------------------------------------------------------- One processes isn’t going to cut it The gift and the curse of node.js is that it makes you address scale right off the bat because there is no resources sharing across processes. It should be M.A.N.E.C. because you aren’t going to build anything of scale or of a reasonable distributed nature without a cache. Is there a better way to cluster? The best part about the Node.js community is that they have been very good at accepting the fact that they didn’t get it right the first time. Ways to improve the language are constantly in debate and releases are pretty regular. The largest anticipated change (v0.11.2) is coming to the cluster module which provides for a round robin approach to handling the incoming request load. Watch the StrongLoop blog posts for good information about what is coming down the pipe.
- Building a stateless backend means that you can deploy almost anywhere. Best of all, you can deploy to elastic clouds, a fancy term used to describe environments that give you more resources under load and then release those resources once the load has subsided. There are many options to choose from. I’ve put a few of them up here. Self managed instances can be a real drain on your team’s productivity. Being able to package and deploy to one entity is critical. It keeps you fault tolerant, scalable, and within budget. Teams can choose to manage their own instances but be wary because most self managed instances focus on a small number of cores and larger amounts of memory combined with different storage models. These are built to fit the traditional web server model. Node.js is different because it can be more efficient with a smaller number of cores.
- In the early days of PencilBlue we didn’t want to break the bank and we were quite ready for an elastic deployment. We went with Linode because of the pricing model. We could take multiple $10/mo instances and a load balancer and run for a fraction of what it would cost in AWS. The diagram above shows what PencilBlue’s production site looked like at launch. We had 1 managed load balancer, two small VMs and one medium VM. We also had backups being taken on the medium server to protect our data. The smaller nodes didn’t need it because they were stateless and we could recreate those pretty easily. The monthly bill was around $70. For those teams that are on a strict budget this is a perfect setup. Small and easily manageable. You want to look for the hosting provider that handles load balancing then take a few of the smaller nodes they have. Optimally, you want to get to a point where you can run in an elastic environment.
- At the end of the day it is up to you and your team to decide which technologies are right for your project. The MEAN stack should be considered a set of guidelines rather than a strict set of technologies meant to operate in unicen. The reality is that for some project they just don’t mesh well. Learn from our experiences and take the time to understand what technologies help to solve your most complicated problems.