Arguably the most important hyperparameter, the learning rate, roughly speaking, controls how fast your neural net “learns”. So why don’t we just amp this up and live life on the fast lane?
Not that simple. Remember, in deep learning, our goal is to minimize a loss function. If the learning rate is too high, our loss will start jumping all over the place and never converge.
And if the learning rate is too small, the model will take way too long to converge, as illustrated above.
Consistency defines the rules under which distributed data is available to users. What this means is that when new data is available (i.e., new, or updated data) in a distributed database, the consistency model determines when the data is available to users for reads. Besides strong and eventual, there are three additional consistency models. These are the bounded staleness, session, and consistent prefix.
Strong
Consistency Model
Guarantees that any read of an item (such as a customer record) will return the most recent version of such item.
Eventual
Consistency Model
When using the eventual consistency model, it is guaranteed that all the replicas will eventually converge to reflect the most recent write.
Bounded
Staleness Consistency Model
With bounded staleness, reads may lag writes by at most K operations or a t time interval. For an account with only one region, K must be between 10 and 1,000,000 operations, and between 100,000 and 1,000,000 operations if the account is globally distributed.
Session
Consistency Model
The session consistency model is named so because the consistency level is scoped at the client session. What this means is that any reads or writes are always current within the same session, and they are monotonic.
Consistent
Prefix Consistency Model
The last consistency model is consistent prefix. This model is like the eventual consistency model; however, it guarantees that reads never see out-of-order writes.
Consistency
for Queries
By default,
any user-defined resource would have the same consistency level for queries as
was defined for reads.
