MongoDB Detective Kit for DBAs

The mongostat utility provides a quick overview of the status of a currently running mongod or mongos instance. mongostat is functionally similar to the UNIX/Linux file system utility vmstat, but provides data regarding mongod and mongos instances.

mongotop provides a method to track the amount of time a MongoDB instance spends reading and writing data. mongotop provides statistics on a per-collection level. By default, mongotop returns values every second.

mongoreplay is a traffic capture and replay tool for MongoDB that you can use to inspect and record commands sent to a MongoDB instance, and then replay those commands back onto another host at a later time.

mongo is an interactive JavaScript shell interface to MongoDB, which provides a powerful interface for systems administrators as well as a way for developers to test queries and operations directly with the database. mongo also provides a fully functional JavaScript environment for use with a MongoDB. This document addresses the basic invocation of the mongo shell and an overview of its usage.

The database profiler collects fine grained data about MongoDB write operations, cursors, database commands on a running mongod instance. You can enable profiling on a per-database or per-instance basis. The profiling level is also configurable when enabling profiling. The profiler is off by default.

The GUI for MongoDB. Visually explore your data. Run ad hoc queries in seconds. Interact with your data with full CRUD functionality. View and optimize your query performance. Available on Linux, Mac, or Windows. Compass empowers you to make smarter decisions about indexing, document validation, and more.

mtools is a collection of helper scripts to parse, filter, and visualize MongoDB log files (mongod, mongos). mtools also includes mlaunch, a utility to quickly set up complex MongoDB test environments on a local machine.

Export dat z MongoDB

Čas od času se mi stane, že potřebuji přímo z MongoDB vyexportovat nějaký konkrétní objekt/dokument, nebo celou sadu objektů splňující zadanou podmínku, do JSONu pro nějaké další zpracování, třeba jen přeposlání někomu dál a nechce se mi použít nějaký komplexní GUI nástroj, nebo nechci hned psát JS kód.

Pak sáhnu po nástroji mongoexport, který je jako binárka přímo součásti distribučního balíčku Monga:


Inuput Outpu Operations Per Second

Obecná horizontálně škálovatelná architektura MongoDB clusteru.

Zvýšení performance IOPS rozdělením dat, indexů a logs na samostatné disky

Novinky v MongoDB 3.4

Na konci minulého roku vyšla nová verze MongoDB ve verzi 3.4, která je nyní ready for production deployment.
What’s New in MongoDB 3.4 white paper

Pěkný článek popisující novinky verze 3.4 o třech částech: 1.díl, 2.díl, 3.díl.


Native Graph Processing

Applications storing data in MongoDB frequently contain data that represents graph or tree type hierarchies. Graph and hierarchical data is commonly queried to uncover indirect or transitive relationships. MongoDB 3.4 offers this functionality via a new aggregation stage called $graphLookup to recursively lookup a set of documents with a specific defined relationship to a starting document. Developers can specify the maximum depth for the recursion, and apply additional filters to only search nodes that meet specific query predicates. $graphLookup can recursively query within a single collection, or across multiple collections.
MongoDB $graphLookup operator for graph processing

Faceted Navigation

Faceting is a popular analytics and search capability that allows an application to group information into related categories by applying multiple filters to query results. Facets allow users to narrow their search results by selecting a facet value as a filter criteria. Facets also provide an intuitive interface to exploring a data set, and allow convenient navigation to data that is of most interest. MongoDB 3.4 introduces new aggregation pipeline stages for the bucketing, grouping and counting of one or more facets in a single round trip to the database. As a result, developers can generate richer and intuitive experiences to help users navigate complex data sets.
MongoDB faceted navigation

Multi-Language Collations

Each language has different rules governing the comparison and sorting of data. In order to create intuitive, localized user experiences, applications must handle non-English text with the appropriate rules for that language. For example, French has detailed rules for sorting names with accents on them. German phonebooks order words differently than the German dictionary. MongoDB 3.4 significantly expands language support capabilities to allow users to build applications that adhere to language-specific comparison rules. Support for collations – the rules governing text comparisons and sorting – has been added throughout the MongoDB Query Language and indexes for over 100 different languages and locales. Each collation can also be further customized to provide precise control over case sensitivity, numeric ordering, whitespace handling, and more.
collation in MongoDB from the documentation

Aggregation Pipeline Enhancements

MongoDB 3.4 continues to extend the aggregation pipeline by adding new capabilities within the database that simplify application-side code, as well as optimizer enhancements that improve performance.
MongoDB 3.4 aggregation pipeline enhancements from the documentation

Decimal Data Type

Decimal128 is a 16 byte decimal floating-point number format. Decimal128 supports 34 decimal digits of significance and an exponent range of −6143 to +6144. MongoDB 3.4 adds support for the decimal data type which represents decimal128 values. Unlike the double data type, which only stores approximations of decimal values, the decimal data type stores the exact value.
MongoDB decimal data type

Visualizing MongoDB Data

Building on its initial release, the Connector for BI has been reengineered to improve performance, simplify installation and configuration, and support Windows. Performance and scalability has been improved by moving more query execution down to the MongoDB processes themselves. Queries and complex aggregations are executed natively within the database, thus reducing latency and bandwidth consumption. The Connector for BI is part of the Advanced Analytics suite available with MongoDB Enterprise Advanced.
MongoDB Connector for BI documentation

MongoDB Connector for Apache Spark

MongoDB Connector for Apache Spark has been updated to support the latest Spark 2.0 release.

MongoDB Zones: Sophisticated Data Distribution

MongoDB zones (superceding tag-aware sharding in earlier MongoDB releases) allow precise control over where data is physically stored, accommodating a range of deployment scenarios – for example by geographic region, by hardware configuration, or by application feature. Administrators can continuously refine data placement rules by modifying shard key ranges, and MongoDB will automatically migrate the data to its new zone.
MongoDB zones

Elastic Scalability

Faster Cluster Balancing & Node Synchronization: The balancer process now supports parallel data migrations. Multiple node pairs can perform balancing migrations simultaneously, significantly improving balancing throughput as nodes are added or removed from the cluster, or as data is redistributed across nodes.
Intra-Cluster Network Compression: MongoDB 3.4 introduces a new option to compress the wire protocol used for intra-cluster communications. Based on the snappy compression algorithm, network traffic can be compressed by up to 70%, providing major performance benefits in bandwidth-constrained environments, and reduced networking costs.

Linearizable Reads

MongoDB 3.4 adds a new readConcern level of “linearizable”. This option confirms the primary replica is still connected to a quorum (majority) of replica nodes before returning results to the client. When used to perform reads against a single document, linearizable read concern provides two guarantees: First, it guarantees that the returned data reflects only writes that are committed to a majority of nodes in the replica set, and therefore will not roll back in the future as a result of a replica set election. Second, it guarantees that the read is not stale. This means that the returned data reflects the last write operation to the document that successfully replicated to a majority of nodes. In order to provide the extra guarantees, using linearizable read concern level will have a significant impact on read latency.
reviewing the linearizable read concern documentation

Expanded Platform Support

MongoDB 3.4 has been ported to the ARM v8-64 bit platform. MongoDB 3.4 has been ported to IBM’s POWER8 and zSeries platforms.

Enterprise-Grade Security for Regulatory Compliance

LDAP Authorization: When configured with an LDAP server for authorization, MongoDB 3.4 will allow user authentication via LDAP, Active Directory, Kerberos, or X.509 without requiring local user documents in the $external database.
Read-Only Views: New in MongoDB 3.4, DBAs can define non-materialized views that expose only a subset of data from an underlying collection.

MongoDB Compass

The latest Compass release expands functionality to allow users to manipulate documents directly from the GUI, optimize performance, and create data governance controls.

Operational Management for DevOps Teams

Ops Manager is the simplest way to run MongoDB on your own infrastructure, making it easy for operations teams to deploy, monitor, backup, and scale MongoDB. Ops Manager is available as part of MongoDB Enterprise Advanced, and its capabilities are also available in Cloud Manager, a tool hosted by MongoDB in the cloud.

Ops Manager Monitoring

Ops Manager now allows telemetry data to be collected every 10 seconds, up from the previous minimum 60 seconds interval.

MongoDB Atlas: VPC Peering

MongoDB Atlas offers the latest 3.4 release (community edition) as an option. In addition, MongoDB Atlas also now offers AWS Virtual Private Cloud (VPC) peering. Each MongoDB Atlas group is provisioned into its own AWS VPC, thus isolating the customer’s data and underlying systems from other MongoDB Atlas users. With the addition of VPC peering, customers can now connect their application servers deployed to another AWS VPC directly to their MongoDB Atlas cluster using private IP addresses.

MongoDB backup

JeduEdu používá jako svůj datastore technologii MongoDB (skvělá dokumentová databáze). Aby dostupnost databáze byla opravdu 100%, data jsou ukládána v ReplicaSetu, který tvoří 3 nezávislé fyzické servery. To znamená, že každý zápis se provádí na primary server a operace zápisu se považuje za úspěšnou, až minimálně jeden další server ReplicaSetu provede zápis u sebe. Skvělé.
Data jsou fyzicky na 3 různých místech. Další výhodou je, že k tomu, aby aplikace koretně stále fungovala stačí, aby fyzicky běžel jen jeden ze serveů. Servery nepadají každý den, ale stát se to může, ale že vám spadnou hned 3 servery je docela nepravděpodobné.

Dalším krokem k zabezpečení dat je jejich pravidelné zálohování. Bez ohledu na kolika strojích ReplicaSt běží, každý den se automaticky provádějí zálohy celé databáze JeduEdu. Opět na všech 3 strojích.

A aby záloh opravdu nebylo málo, pak čas od času spustím zálohování ze svého Macu, takže mám kopletní zálohu úplně mimi produkční prostředí a jsem schopen celou aplikaci, včetně backendu během okamžiku rozjet úplně někde jinde na něčem jiném.

Odzálohovat Mongo je neskutečně jednoduché a dá se dělat za reálného provozu bez nutnosti odstavení jakéhoholiv ze serverů v ReplicaSetu:

Výhodou ReplicaSetu je, že zálohovat můžete tak, že zastavíte některý ze slave serverů a odkopírujete celý adresář s databází na disku. Po nahození takto „zazálohované“ databáze můžeze slave server zase nahodit a on si automaticky sám dotáhne vše o co přišel po dobu co byl down. Super.

MongoChef GUI pro MongoDB

MongoChef je skvělý nástroj pro práci s MongoDB. Jedná se o aplikaci pro Windows, MacOS i Linux, která vám pomůže jednodušeji ovládat vaše MongoDB databáze.
Až do teď jsem prováděl všechny úkony jen skrze mongo shell. S MongoChef je práce mnohem jednodušší a rychlejší, vůbec v případě, že hledáte v datech, kde je jejich vizualizace opravdu přínosem.
MongoChef má podporu pro MongoDB replica sety a také pro agragace. Super.

Pro nekommerční účely je zdarma. Verze Professional pak stojí 99 USD a verze Enterprise stojí 349 USD. Rozdílem v nekomerční verzi a Professional je jen v míře podpory. Nekomerční verze nezvládá Kerberos, LDAP a MongoDB Enterprise. To vše zvládá až verze Enterprise.

Super na MongoChef je podpora všech operačních systémů, pěkné grafické prostředí aplikace a opravdu přehledná vizulaizace dat.

Alternativní aplikací pro práci s MySQL je Sequel Pro

MongoDB a NodeJS

Using MongooseJs in Node.js and MongoDB
MongoDB blog: about Mongoose
Web Dev – 5610 – MongoDB And Mongoose
OK Coders: Mongoose ~ Using MongoDB in Node

Dnes jsem po pádu jedno z mých serverů opětovně rozchodil web pro

MongoDB je moderní databázová technologie, kterou používám u všech novějších projketů. V podstatě jsem tak úplně nahradil MySQL. Asi nejpádnějším argumentem pro tuto výměnu byla horizontální škálovatelnost Monga a navíc jeho dokumentový přístup k ukládaným datům.

Dokumentová databáze

Žádné přesně definované sloupce pro řádky v tabulkách, ale dokumenty se svými vlastnostmi.


No a pokud jeden server nestačí, jednoduše přidáte další.

© 2017

Theme by Anders NorénUp ↑