NOTICE: NEW FORUMS

We are excited to share with you our brand-new and improved Disruptor Beam forums!

We've chosen new forum software that brings added functionality to improve your experience, including increased spam management. Your username details have been carried over, so you should be able to login with your old forum details without issue. For our veteran community members, you'll see that we have many of the thread categories you’re used to, so we hope you'll feel right at home there.

Our new forums are located here: https://forum.disruptorbeam.com/stt/

 
   
Gauntlet: featured skill inquiry
Posted: 02 March 2017 11:48 AM   [ Ignore ]  
Federation Council
RankRankRankRankRankRankRank
Total Posts:  5993
Joined  2013-02-08

Hi all,

We are starting this thread to address the concerns that the featured skill might change during a specific Gauntlet run.

We believe this to be a display issue but are more than willing to dig a bit deeper into it.

To do so we need specific details:

- date/time you started the Gauntlet
- featured skill for that Gauntlet
- date/time when you believe the featured skill changed, and what skill you saw
- your DBID (this can be provided to me via PM referring to this thread)

Add any other detail that might be relevant.

Thank you!

 

 Signature 

Shan
Player Experience Specialist, Disruptor Beam

Profile
 
 
Posted: 02 March 2017 08:24 PM   [ Ignore ]   [ # 1 ]  
Avatar
Lieutenant
RankRankRankRank
Total Posts:  341
Joined  2016-11-30

From another thread:

I can confirm, if I refresh with merits, guaranteed every time one of the skills presented to me is displayed as the featured skill if I go back to the rankings (and that skill will more often than not be in the rounds after on wins/losses).

A straight win or loss doesn’t appear to change the featured skill from what I’ve seen.

I’ll make a note of the data you’ve requested next time I do a merit refresh, with screenshots.

 Signature 

/SSR/Ninpo

Fleet for Serious Trekkers

Profile
 
 
Posted: 02 March 2017 08:32 PM   [ Ignore ]   [ # 2 ]  
Avatar
Lieutenant
RankRankRankRank
Total Posts:  341
Joined  2016-11-30

I don’t have exact times, but:

My Gauntlet started with a featured skill of DIP.  I’ve seen SEC and CMD and ENG featured, currently it’s:

and the list of opponents:

Time remaining (ish from time of post):

You can tell after a merit refresh followed by a loss/win which skill got flipped to because it appears twice in a row.


Edit:

I won against Picard in the above image:

Oh look, MED again.

I then refreshed and got DIP back as the featured skill and beat a Mirror Kirk in a CMD+DIP match (then completely forgot to grab screenshots, apologies).

I’ve since tanked a match and:


I’d repeat this, but FIFTY MERITS IS EXPENSIVE.

 Signature 

/SSR/Ninpo

Fleet for Serious Trekkers

Profile
 
 
Posted: 03 March 2017 12:18 AM   [ Ignore ]   [ # 3 ]  
Avatar
Lieutenant Commander
RankRankRankRankRank
Total Posts:  584
Joined  2016-01-17

Shan, I’ve recorded my Gauntlet run today and from that have identified the problem.

Whatever the featured skill is, it then selects two skills for the current contest, with a bias toward the featured skill. This bit actually works.

However, of these two skills, the first one selected *becomes* the featured skill, every single time, and so the next pair of contest skills is selected based on the bias of the new featured skill. Anyone can test this themselves at any time.

Because of the bias, the featured skill is likely to choose itself, so you tend to see small runs of say, 3 SCI in a row, then 3 MED in a row, or whatever. So at the very beginning of the Gauntlet it would look like everything was normal because you’re likely to get several instances of the featured skill in a row, which is what you’d expect.

To fix this problem, all you need to do is, you know, not change the featured skill. Whatever bit of code sets the first of the two contest skills, is also setting the featured skill. Fix that line of code and the problem will go away. This could easily be caused by someone forgetting to declare “skill” (#1) as a local variable in the contest method, and therefore it’s setting the “skill” variable in the next closest scope. So it’s actually an easy fix.

Get the code monkeys on that and let me know how it goes. grin

Profile
 
 
Posted: 03 March 2017 08:12 AM   [ Ignore ]   [ # 4 ]  
Avatar
Lieutenant
RankRankRankRank
Total Posts:  341
Joined  2016-11-30
Philip Broughton-Mills - 03 March 2017 12:18 AM

Shan, I’ve recorded my Gauntlet run today and from that have identified the problem.

Whatever the featured skill is, it then selects two skills for the current contest, with a bias toward the featured skill. This bit actually works.

However, of these two skills, the first one selected *becomes* the featured skill, every single time, and so the next pair of contest skills is selected based on the bias of the new featured skill. Anyone can test this themselves at any time.

Because of the bias, the featured skill is likely to choose itself, so you tend to see small runs of say, 3 SCI in a row, then 3 MED in a row, or whatever. So at the very beginning of the Gauntlet it would look like everything was normal because you’re likely to get several instances of the featured skill in a row, which is what you’d expect.

To fix this problem, all you need to do is, you know, not change the featured skill. Whatever bit of code sets the first of the two contest skills, is also setting the featured skill. Fix that line of code and the problem will go away. This could easily be caused by someone forgetting to declare “skill” (#1) as a local variable in the contest method, and therefore it’s setting the “skill” variable in the next closest scope. So it’s actually an easy fix.

Get the code monkeys on that and let me know how it goes. grin

I’m willing to bet this is actually what’s happening, that the selector flips the featured skill each time and of course, given the weighting they have set on the actual featured skill, it gets set more often than others.  But as you say, the in game experience is horribly skewed because when the featured skill changes, you can tell, as there’s a mini run on that skill in game until it changes again.

It’s NOT a visual bug.  I wouldn’t be able to tell from the opponents being presented over 2-3 turns that it has changed, if it was simply visual.

 Signature 

/SSR/Ninpo

Fleet for Serious Trekkers

Profile
 
 
Posted: 03 March 2017 08:54 AM   [ Ignore ]   [ # 5 ]  
Avatar
Lieutenant Commander
RankRankRankRankRank
Total Posts:  584
Joined  2016-01-17

Here’s my actual data from this morning. I did not try refreshing with merits, so that may behave differently.

Gauntlet started: 2017 March 03
Featured skill: CMD (traits: Gambler Bajoran Pilot)
Skill changed: Every contest.

UTC / FEATURED / PAIR
04:08 CMD SEC+MED
04:10 CMD CMD+MED
04:12 ENG ENG+SEC
04:14 MED MED+CMD
04:15 ENG ENG+MED
04:16 MED MED+SEC
04:21 MED MED+ENG
04:22 MED MED+SEC
04:23 MED MED+DIP
04:24 CMD CMD+MED
04:25 SEC SEC+SCI
04:27 SEC SEC+CMD
04:28 CMD CMD+MED
04:30 SEC SEC+MED
04:32 MED MED+DIP
04:34 DIP DIP+ENG
04:35 SEC SEC+MED
10:52 CMD CMD+MED
13:28 CMD CMD+ENG
13:30 CMD CMD+SCI
13:31 ENG ENG+DIP
13:32 ENG ENG+CMD
13:33 CMD CMD+SEC
13:35 SEC SEC+DIP
13:36 CMD CMD+SEC
13:37 MED MED+SCI
13:38 ENG ENG+MED
13:40 DIP DIP+SCI
13:41 SEC SEC+SCI
13:42 DIP DIP+MED
13:43 CMD CMD+SEC
13:45 DIP DIP+SEC
13:46 SCI SCI+ENG

TOTALS:
CMD 12
DIP 8
ENG 9
SEC 14
MED 15
SCI 8

Clearly CMD did not come up a greater number of times than the other skills.

Profile
 
 
Posted: 03 March 2017 12:37 PM   [ Ignore ]   [ # 6 ]  
Avatar
Lieutenant
RankRankRankRank
Total Posts:  341
Joined  2016-11-30

04:16 MED MED+SEC
04:21 MED MED+ENG
04:22 MED MED+SEC
04:23 MED MED+DIP

They’re the ones that drive me mad.

 Signature 

/SSR/Ninpo

Fleet for Serious Trekkers

Profile
 
 
Posted: 03 March 2017 02:47 PM   [ Ignore ]   [ # 7 ]  
Avatar
STT Command
RankRankRankRankRankRank
Total Posts:  3840
Joined  2013-05-15

Right, so:

My understanding of the issue is that the featured skill indicated on the info button changes as the gauntlet is refreshed. Some players believe that this isn’t just a display issue, but also biases the skills they receive on their refreshes.

I ran some tests on my own account, where I reproduced the display issue, where the featured skill on the info button changes based on the first skill that is shown after the refresh. This is the known display issue. After every refresh I queried the database to find the exact state of my Gauntlet, including the featured skill of the gauntlet. I then recorded three hundred Gauntlet matchups for my account, using the same in-game actions that our players have used (refresh, see featured skill change, etc.). Here are the results!

After 300 matchups I found the distribution of skills to be:

Skill 1: 11.71%
Skill 2: 28.09%
Skill 3: 16.39%
Skill 4: 13.38%
Skill 5: 15.05%
Skill 6: 15.38%

Where Skill 2 was the featured skill inherent to the gauntlet itself. This matches close-ish to the expected distribution:

Actual odds: Featured skill 28.57%, all other skills 14.28%.

This also mirrors the distribution I found yesterday when I looked at a snapshot of active Gauntlets of the same type:

Snapshot time: 11:30am ET
Total number of active Gauntlets reviewed: 17,500
Featured skill: Diplomacy.
Command %: 14.95%
Diplomacy %: 29.22%
Engineering %: 14.14%
Medicine %: 13.74%
Science %: 13.69%
Security %: 14.23%

Some of the questions have been whether the refresh biases the following skills, and whether the displayed featured skill biases the following matchups. I can say that after 300 matchups this does not appear to be the case. Normal random distribution does have streaky portions; at once point, I have seven matchups that all began with Diplomacy. Still, in the end, they balanced out. What seemed unbalanced at 30 attempts became closer at 170, and closer still at 300.

This makes sense, because what your client displays is irrelevant when assigning the featured skill. The featured skill is inherent to the gauntlet archetype you’re currently in. The archetype is the server-side definition of what your gauntlet contains, like the loot, the featured skill, and featured traits. That value is referenced in the database itself and assigned at the time you enter the gauntlet. I specifically looked to see if this changed by querying the production database each time after I refreshed, and each query returned the same (correct) gauntlet archetype.

If needed I can run an additional 200-300 atempts to generate results closer to the ideal, but at this time I still believe this is a display issue only.

If you have any further questions, please let me know!

 Signature 

Nod
Product Manager, Star Trek Timelines

Profile
 
 
Posted: 03 March 2017 03:02 PM   [ Ignore ]   [ # 8 ]  
Avatar
STT Command
RankRankRankRankRankRank
Total Posts:  3840
Joined  2013-05-15

What’s being brought up here isn’t necessarily about whether the system works; I think it is important to note that the system works, that the featured skill changing is indeed a display issue, and a display issue only; and that the expected distribution does occur over time. But what the feedback may be asking about is whether this is apparent to players, in other words, whether the weighting towards the featured skill comes up often enough for it to feel like it’s meaningful. That’s a very different discussion and that kind of feedback is really useful.

However, for this specific concern, where players believe that the displayed skill is biasing the refreshes, that doesn’t appear to be the case. I believe we can still say that the issue is purely a display issue, and your featured skills do indeed show up more often than the other skills. Whether they show up often enough, and whether the weighting is heavy enough, is something we’ll continue watching and gathering feedback on.

 Signature 

Nod
Product Manager, Star Trek Timelines

Profile
 
 
Posted: 03 March 2017 05:57 PM   [ Ignore ]   [ # 9 ]  
Avatar
Lieutenant
RankRankRankRank
Total Posts:  341
Joined  2016-11-30

Nod, if you’re needing 300+ runs for the bias to bear out, surely that should suggest something is amiss?  After all, the end goal here is for the end user to perceive a bias to the featured skill, yes?  That’s the point of having it, after all?  Otherwise you’re saying “you may or may not see more of this skill, who knows!”

We’re not playing a statistics simulator, or taking a probability class.  Statistics mean nothing to the individual, you’re handling a game and a user experience here.

User experience.  The current UX does not come even close to “a bit under a third of the time you get the featured skill” because we don’t get to play long enough for the variance to even out, mostly because of the weird back-to-backing of the “new” featured skill that’s been demonstrated to you in this very thread.

It’s irrelevant that if you are able to freely punch things thousands of times in your play the game for free simulator because you’re not testing the user experience.  The user experience, currently, is that the weighting is clunky and in our limited window of access, doesn’t bear out the claims.  You can’t say “everything is great over a thousand runs” and then set the environment up to give us ten.

Put the We’re-not-Wrong-Number-Masher-o-Matic down and stick your head in the game with an average amount of dilithium and merits and a realistic mix of crew and actually play it, see what happens on refreshes and see the correlation of back to back repeat skill opponents matching up with the featured skill.  Get a feel for what you’re actually giving your customers and their experience.

I’m not here for a statistics class, I’m here to use a product targeted at end users.

There’s a saying in CS circles:  In theory, there’s no difference between theory and practice.  In practice, there is.  Spinning hundreds of Gauntlets in the lab is far from actual practice.

 Signature 

/SSR/Ninpo

Fleet for Serious Trekkers

Profile
 
 
Posted: 09 March 2017 03:46 AM   [ Ignore ]   [ # 10 ]  
Avatar
Lieutenant Commander
RankRankRankRankRank
Total Posts:  584
Joined  2016-01-17

The recent Gauntlet update claimed to fix this issue, but my featured skill is still changing. (Started new Gauntlet March 9th. Can see the other updates such as increased merits.)

Image Attachments
Screenshot_20170309-082055.pngScreenshot_20170309-091814.png
Profile
 
 
Posted: 09 March 2017 01:01 PM   [ Ignore ]   [ # 11 ]  
Federation Council
RankRankRankRankRankRankRank
Total Posts:  5993
Joined  2013-02-08

The display issue has not been fixed yet.
It is not part of the changes mentioned in our build notes.

 Signature 

Shan
Player Experience Specialist, Disruptor Beam

Profile
 
 
Posted: 10 March 2017 11:28 AM   [ Ignore ]   [ # 12 ]  
Avatar
Ensign
RankRankRank
Total Posts:  205
Joined  2016-07-23
Shan - 02 March 2017 11:48 AM

Hi all,

We are starting this thread to address the concerns that the featured skill might change during a specific Gauntlet run.

We believe this to be a display issue but are more than willing to dig a bit deeper into it.

To do so we need specific details:

- date/time you started the Gauntlet
- featured skill for that Gauntlet
- date/time when you believe the featured skill changed, and what skill you saw
- your DBID (this can be provided to me via PM referring to this thread)

Add any other detail that might be relevant.

Thank you!

Started: Mar 8, 19:00 EST (will end Mar 10, 19:00)
Featured skill: SEC
Change: ~ Mar 9, 19:00 EST
New featured SKill: MED

And I can confirm, this is NOT a display issue.

I picked my team based on the featured skill of SEC.
I have only 1 MED crew, 5* Holographic Dr.

Now suddenly, MED is coming up every other match up .. and SEC is not often seen. :(
I did well in the first phase, was in top 10 fairly easy .. but now I’m struggling ...
:(

Hmm, just went back in ... changed again.
so sometime between 07:00 EST and 12:00 EST ...
it changed from MED .. to SCI. O.o

(or and Locutus is still got too many horseshoes ... x3 in a row he’s beat my Paris when I’ve had better stats .. *sigh* )

wow .. and it changed again:
time is now: 12:13 EST.  so there’s a 15 min window.
it’s now DIP.
*facepalm*
I was wondering why I wasn’t seeing any SCI missions come up ... that explains it ... *sigh*

 

Profile
 
 
Posted: 14 March 2017 07:50 AM   [ Ignore ]   [ # 13 ]  
Avatar
Ensign
RankRankRank
Total Posts:  238
Joined  2016-11-24

The current gauntlet I’m in changed the featured skill from SEC at selection to ENG when playing.

I’m not so sure about it only being a display bug as I’m seeing quite a few eng matchups as opposed to security ones but I’ll keep an eye on it and post my findings here (it won’t be for 300 refreshes though).

Profile