Theorycrafting on Feral: AMR, SimC, and Pooling

Hey all, I’m Swol, the main theorycrafter for Ask Mr. Robot (AMR).  Most of you don’t know me by name, but I’ve been posting theorycraft here and there lately for interesting topics I run across when I have something useful to say that isn’t common knowledge yet.

Recently, some Feral Druid theorycrafters suggested that AMR was bad for feral druids, in general. When we asked them why they had that opinion, a big topic that came up was energy pooling.

In this article I am going to examine the theory behind energy pooling for feral druids and go into detail on how to test that theory. To do this, I will use the default Action Priority List (APL) found in the nightly SimC build and explain how it works.

And, I will also use an APL that I have developed which uses a different structure and, specifically, does not pool energy before using Rip. After this analysis, I will then provide a detailed comparison between the SimC simulator and AMR simulator for feral druids so that theorycrafters can comfortably trust and understand both tools.

Continue reading

6 Biggest stat weight questions, answered!

1. Do I need custom stat weights?

Our default stat weights are similar to those you find in guides – they are a starting point for casual players. You don’t want to spend a ton of time outside of the game theorycrafting? No problem, use our defaults, or enter in defaults from your favorite guide by clicking the green ‘view/edit’ button above the stat section.

If you use default stat weights, or weights from a guide, the results suggested to you will very likely sim differently. If you’re going to be simming to verify results, you definitely want custom.

For those of you into min-maxxing, yes! Custom weights are tailored to your specific character. They take into account your talents, relics, legendaries and so on. All of these things affect your stats, so that’s why min-maxxers get custom weights.

2. Should I use Ask Mr. Robot (AMR) or SimC?

It depends on what you want to do. If you’re simming just to find your DPS, both programs tend to be accurate, and both have very smart people working on them. Most classes collaborate across the platforms to make sure both are accurate as well since it’s in everyone’s interest to make sure both work. Anyway, pick your favorite.

However, when it comes to stat weights, SimC does a subset of what AMR does. One of the things we really focused on when we built our simulator was improving the accuracy of stat weight generation by using a statistically accepted practices. We also built the stat weight functions to integrate with gear ranking. Let me explain:

  • AMR looks at 1000s of data points over real gear stat budgets. We then are able to apply statistical models to this to find a ‘slope’ (how much your DPS increase with each stat). That is your stat weight.
  • SimC looks at 2 data points, one above your gear and one below your gear.  Then a line is drawn between those 2 points to find the slope. Because of this, the slope changes depending on the 2 data points that were chosen.
  • AMR looks at how stats interact. SimC looks at each stat in isolation with the stat scaling function, missing the interactions. However, for those of you familiar with SimC reforge plots, those do take a look at stat interactions, and then you can manually adjust stat weights based on your interpretation.
  • AMR sims all trinket, legendaries, and things with procs and bundles that with the stat weight report so that we can rank those items from sim data. SimC’s stat weights don’t do this.

This is not an AMR vs SimC battle, like I said, it is a great program. One feature not being as robust doesn’t make it less worthwhile to use as a simulation tool. Same goes for AMR.

3. Why do Mr. Robot’s trinket rankings look different than from a guide?

Trinket lists you find in a guide are for casual players just looking for a quick estimate to eyeball what might be best. Min-maxxers will want to get a list custom tailored to them, as legendaries, talents, and your specific stats will affect how trinkets perform.

You can use any simulator to generate a custom trinket list based off your current gear. If you get custom stat weights from Mr. Robot, the trinkets are simmed custom to you, and that data is used to rank the trinkets directly from those simulations.

To repeat: Mr. Robot ranks all trinkets, and any item with a proc, directly from simulation data. Get custom weights to make sure your trinket lists are based off custom sim data.

If people say our trinkets lists aren’t reliable, they probably don’t have a custom set of weights. Tell them to hit me up and I can walk them through that setup.

4. Why do gear suggestions sometimes sim less DPS? Is Mr. Robot broken?

The goal is that all gear recommendations are within 1% of their actual simulated DPS. There is a tradeoff of speed for accuracy. Mr. Robot ranks an entire list of gear for you in < 1 second, and gets you within 1% of the actual DPS. For example, for Best in Bags, Mr. Robot sifts through several millions of combinations of gear that you own, and does this in about 1 second. If you simulated every one of those combinations, it would take about 12-24 hours to run.

The reason we put that green ‘simulate’ button next to all of your ranked gear lists, is to let you play with it. Want to swap in one other item and sim it at a 0.05% margin of error? Do it. It’s fun. Have fun.

That being said, if you find something WAY off, definitely let us know so we can look into it.

5. Can you simulate tanks and healers?

Yes! Although this is currently very controversial as there are 2 camps who feel strongly in either direction. We like simulation, but that’s probably obvious. Other people don’t and aren’t afraid to tell you that. It’s your call!

Tanking: we improved on the model that SimC’s had over the years past, with super smart people pushing the way, like Theck (TMI). He took a break from wow and we picked up the torch to continue pressing forward on tank simulations.

  • We let tanks die! (SimC gives you infinite health).
  • We built our NPS metric to track death chance. So the lower your chance to die, the higher your NPS is. That’s our ultimate measure of a tank’s tankiness. We don’t factor in DPS right now.
  • We give you a team of healers that act like ‘real’ healers. If your health is high, healers hot you and use small heals. As your health gets lower, they use bigger heals. If your health is really low, they start using instant-heals.
  • We built in boss events that you can anticipate. If the boss is going to do a giant strike-of-face-smashing, we let you save your big cooldowns for that event. Just like you would in a real raid.

We think the tanking is really robust. Our developer plays a tank (if that makes you feel any stronger about it?). If you think there are other areas to improve it, hit us up.

Healing: SimC doesn’t sim healers, so we’re pushing into new territory here.

  • Our first (and current) model, assumes you have only the number of healers on your team that you need. Meaning that you don’t have time to stand around and not heal (if you did, often you can replace a healer with a dpser).
  • This means mana is important. Our simulator factors in running out of mana vs doing more healing.
  • We don’t currently have a model that looks at periods of burst healing and then no healing. We’ll be adding this in soon.

6. What is this “Machine Learning” I keep hearing about?

It’s a better way to rank gear.  Read the full explanation on Machine Learning, written in a fun, cheeky way, that us non-math PhDs can understand, promise!

The short story is this: stat weights are basically a priority order. If Crit has a weight of 9 and mastery has a weight of 8, crit will always be favored. In a perfect world, these stat weights would find gear with only crit (and your main stat, like Int or Str). So you’d have a ton of crit and nothing else. But usually, you want some balance, or some ‘ratio’ of stats. This is one of the main reasons you see people recommending ‘caps’ – the point at which the interaction between 2 stats changes.

Machine learning knows all of this. It knows how stats interact. It knows if you can’t reach some threshhold, that maybe an entirely different setup is better.

And the biggest thing you’ll probably like: machine learning adapts as you get gear. That’s right, you no longer need to re-run stat weights every time you get a new piece of gear. Machine learning has all of the data, and changes with you. Except… you’ll want to run a new gearing strategy if something major changes: talents, legendaries, or relics.

How to generate a custom gearing strategy

Follow the top half. Then if you want to use machine learning, the bottom section shows you how to set that up. Click the image to make it larger, then you might have to click it more more time to zoom. Don’t try to read the tiny print, I hear that is bad for your eyes.

How to generate stat weights

 

 

 

Simulator Client Roadmap

Over the next several weeks we will be releasing updates to the simulation client to support a lot of cool upcoming features. We will do this in stages so that we have an opportunity to test out each stage and work out any issues that people might encounter.

General Features

These new general features are available as of this post, a few that have been there and a few you may not have noticed:

  1. The client auto-updates now. Once you download it and start it, you can theoretically let it run forever.
  2. There is a new “Clear Queue” option in the menu on the client program. Use this to cancel all running and pending simulations. Useful if you accidentally started a long-running simulation with the wrong settings, for example.
  3. As of 2/6, the client uses a different mode that can more fully utilize most CPUs, but uses a bit more memory. Should be faster for most people.

Stage 1: Multiple Clients

Released 2/6! You can now connect more than one simulation client at once, and distribute long-running simulations over multiple computers. This is definitely a power-user feature, so don’t worry about it if you only have one computer or don’t do long simulations.

For those interested in pumping out big custom batches or speeding up gearing strategy runs: you will first need a premium account to run multiple clients concurrently. (This is the same premium account that gives you access to all the other optimizer features like Best-in-Bags and the Upgrade Finder, so most power users are already covered.)

Here are a few tips for getting the most out of the simulation client:

  1. Running a single client on more than 12 threads usually won’t speed things up that much. The only scenario where this tends to be a gain is if each of your cores is slow (an old processor), or you are running really low margin of error simulations (0.1% or less). HINT! Running simulations under 0.25% margin of error is almost always a waste. There are almost no statistically sound reasons to ever go down to 0.1% or less. We have the option in case you come up with an edge case though.
  2. You can turn an extra client on even while a long batch is already running — it will immediately start helping out.
  3. You can turn off a client while it is running as well — the work it is doing will be picked up by another client. Don’t overuse this feature though… there is a limit on how many times it will retry work that gets cancelled (three tries right now). We also don’t recommend stopping clients during gearing strategies right now… they are a bit more complex and may not be as fault tolerant. A future update will improve on this.
  4. If you have a big CPU (8+ physical cores, usually hyperthreaded to 16 logical cores these days), you can sometimes get better performance by running two or more clients on the same machine. To do this, you need to copy the client to multiple folders on your machine and start each separately. (Trying to launch the same executable multiple times will give an error.) For example, try running two 8-thread clients on an 8-core hyperthreaded machine, or three 8-thread clients on a 12-core machine.
  5. You can only connect 25 clients at once. This restriction may be lifted once we test out the system some more.

Stage 2: AMR Global Network

Coming soon! Our goal is to build a global network that uses your idle CPU time to tackle all these hard theorycraft and gear ranking questions! More details forthcoming as we get closer to releasing it. And it will of course be an opt-in feature, never turned on by default.

Stage 3: Automated Gearing Strategies

Once the multi-client features and global network are solid, we want to build features that allow you to automate creating gearing strategies for your characters. Still kicking around ideas, but the idea is to give you an almost zero-maintenance way to keep your gearing strategies up to date when your character changes significantly, and suggest customized simulator settings to make the most of our powerful gear ranking features.

Nighthold is Live!

Here’s what you need to know!

Hotfixes went live in-game today 

  • Simulator: fully updated (though we are doing our routine checks to verify information. Tweaks will be happening over the next few weeks)
  • Stat weights: while updated, these may be tweaked over the next few weeks as we compare rotations to actual logs.
  • Legendaries: The 940 versions are now live
  • Trinkets and set bonuses: While we have game data on these, we like to verify everything about them against real logs. We are starting that process today and it might take a couple of weeks to complete. We do this to assure accuracy on our site.
  • Nighthold bosses: We’ll be adding Nighthold bosses to the simulator over the next few weeks. We’ll be killing bosses this week and working on getting those scripts into the simulator. If you’re some crazy awesome person who already made one, send it our way!

Continue reading

Patch 7.1.5 Updates on Mr. Robot

7.1.5 is a pretty big update (read official patch notes).

Our updates are live, with a couple of notes: 

All classes have updated weights. However! We are still adding really cool parameters to test Surrender to Madness variations and plan to have some of that published by the weekend.

Tanks have been updated, though we still have some tweaks to do for harder hitting nighthold bosses.

It might take us another week or two to finish up final in-game testing and comparing things to logs, in order to verify the accuracy of all data. In addition, after we compare simulations to Nighthold logs, we can tweak rotations to make sure they are back to our gold standard of quality. Just like our Ursoc simulations, which have been matching logs really well.

As always, if something seems odd, ping us (discord / forums). We try to get bugs fixed within 24 hours.

Here are the major things that affect your gear:

  • Changes to your class
  • Secondary stats scale differently now (which we automatically handle)
  • New neck enchants that are more affordable and ready in the optimizer
  • New legendaries that will be pickable in the gear lists. These are also updated in the simulator, but we haven’t finished triple checking them against logs.
  • Crafted items have more upgrade levels, which you can tweak in the filters menu
  • Some changes to existing legendaries and other items, which you can select in the gear lists
  • Simulations: classes will be updated to sim, including tanks and healers. Nighthold boss scripts will be ready shortly after nighthold’s launch. For now use Ursoc for single target and Mythic+ for AoE.

Things to note / things that still need work:

  • Shadow Priests: Surrender to Madness rotation needs tweaking after new changes.
  • Tanks: all weights need to be re-run for a harder hitting boss. The first pass had boss attacks that weren’t doing enough damage, so tanks weren’t dying. That makes stamina look low. But tanks will likely be dying in Nighthold, so we’ll have them updated before you jump in that raid.
  • Protection Warriors: rotation needs some work for the Vengeance and Devastator talents
  • All classes: rotations will be tweaked to squeeze out a little more dps here and there over the next few weeks as we get real game data and players give feedback based on real-game play.

Continue reading

Stat Weights vs Machine Learning

We’re big into ranking gear at Ask Mr. Robot. We probably like it more than WoW itself. OK… almost more than WoW. We like WoW more because it lets us have fun ranking gear.

We released an update today with a beta version of a new way to rank stats on gear that we call Machine Learning. It replaces stat weights. Stat weights are dead to us. OK not really, I was being a bit dramatic. You can still use them on the site like you always did, and they will still work like they always have, forever. But machine learning is way better and way cooler. Read on to learn why!

NOTE: To use the new method during the beta, you must run your own custom gearing strategy simulation, then check “Use Machine Learning” on the report that it generates, and then press “Save and Use”. You will know that it is in effect because you will see a “stat goal” next to each section on the gear optimizer (Gear Explorer, Best in Bags, etc.). If you encounter any bugs or issues, let us know immediately! It is a beta! Continue reading

NPS – A more robust tanking metric

(Written by our developer, Yellowfive)

The first version of this was posted on Nov 16, 2016 for feedback. Based on some good conversations with the community, we have updated the metric and this post to reflect those changes. (Dec 1, 2016). The main updates are related to self-healing and stamina.

NPS

NoPeS. That’s what tanks do: they nope you. Want to hurt me? Nope. Need some healing to stay alive? Nope, I got this. Can I hit you for 2 million damage please? Nope, let’s make that 1 million.

In a sentence: “NPS” measures how good you are at reducing the size of each incoming hit, and how good you are at sustaining yourself without requiring external healing. (Both of those things are given equal value. Higher is better.)

For the sake of ease, let’s say the “N” stands for “negation” in this discussion. Continue reading

AMR Update Nov 16, 2016 at 12:15AM

Here’s our latest update:

  • iLevel updates to Trial of Valor and Legendary items
  • Hotfixes to classes (like surrender to madness for shadow priests) are live in the simulator
  • Mythic+ gearing strategies for DPS specs are available in the ‘gearing strategy’ (stat weight) picker
  • Update to the optimization algorithm which should be mostly invisible to users, but will fix up a couple of bugs people have reported (like equipping 2 of the same legendary ring). This will also make stat caps work nicer (the caps will be obeyed more rigidly)
  • NPS reporting tweaks (how it splits damage mitigation up between abilities). This is a reporting view change only, it doesn’t affect stat weights.
  • Addon is updated to auto-log Trial of Valor

NPS Blog Post is now up.

Karazhan Patch 7.1 Gear & Stat Weights

Here are the Patch 7.1 updates:

  • New gear is added to the gear lists, upgrade finder, and simulator
  • All new trinkets are implemented – and remember, all trinkets are ranked directly from simulations to ensure accuracy. After logs show up tonight, we will be doing a pass that compares all new trinkets to logs to triple-check that they are in tip-top shape.
  • Crafted level 8 is now added to the filters
  • All class and spell changes have been updated in the simulator and theorycraft wiki (the best source for spell data) – link to patch notes
  • NOTE: The chest pieces for each armor type that improves the power of your trinkets is not yet implemented. There is no spell data or data in game with how (and how much) it improves the trinkets. As soon as we get that information, the chests will be updated on our gear lists and simulator. (Example chest)

Stat weights are updated

All stat weights for each class/spec are also updated to reflect the class & spell changes for patch 7.1. All of those changes are verifiable on the theorycraft wiki. Continue reading

How to Compare Items with simulations

The most popular question right now:  What item is better?

Our item rankings answer that question, and it does so really well if you generated custom weights for your character with our simulator.

But how nice would it be to see the actual damage differences, in addition to the ranking? It would be pretty nice, wouldn’t it. Continue reading

« Older posts

© 2017 Mr. Robot's Blog

Theme by Anders NorenUp ↑