QA and better documentation on releases
#1
Feel free to move this to the correct area if required.

I am not trying to start a flame war etc just bring to attention some things I have noticed and touched on.

With the recent released a few bugs have slipped though.

Youtube Videos,
Moderation Logs,
The latest bots spider comma

These have been simple things which with easily overlooked 
or easily have been over sighted in QA.

A simple solution is to release in stages to main testers and developers as a inner circle then the general community same way Microsoft does releases like there insider and feature updates.
Or have a beta period like other software does (Xenforo does this for example) where a release is labeled in the end as .bta or something so 1.8.16.bta that people can publicly test to catch these bugs before a wider release is pushed.

This will stop all these smaller breaking bugs and allow faster development and input from the community on changes and potential issues.
I would even run a dedicated beta forum to help find and report these bugs for the wider released versions as well as many of us here would surely like to do.



As well with the recent change to the login template which needs a change on every custom theme to get that working again,
I feel this wasn't highlighted enough and was missing from the release notes.
Also if It could have been held until 1.9 that would have been better than pushing out a theme breaking change when the whole theme system is being redone.

I made a thread detailing the code change for the login which seems to be the official link to and public notice on this forum for any question about the issue.

MyBB 1.8.16 Released — Security & Maintenance Release
This didn't mention it it only mentioned,
If you use the login_attempt_check() function, its signature has changed
This refers to max login attempts.

In the 61 issues resolved there was nothing listed about a breaking or critical change to the login template in the titles.

In the Release Notes as per the blog the only message was about the max login attempt change as quoted below.
Note: CSS appears typod and should be changed to 'Template and theme files' as CSS is the styling sheets when the changes were in the template code instead.
A confusion point for many.


Quote:This update includes compatibility fixes for database engines and recent PHP versions as well as performance and global security improvements. Note that the theme’s CSS files may need to be updated. If you use the login_attempt_check() function, note that its signature has changed.

The above should also have read as a extra:
The login system has changed slightly to prevent a security breach so your member_login xx and xx templates need changing with this updated code <code here>

The 6 Security vulnerabilities addressed doesn't list the change,

In the file and template changes the templates section the login code there were 47 changed templates alongside the 97 changed files and 3 removed files.
Also there is no way to click on the files in the changed files list to see just the changes to each file and template before updating.

The main login template was listed member_login which is what change that was there were other places I believe the new code was used one the portal wasn't changed and was still bugged.

Below the changed template is just the MyBB footer.

So nowhere was a public notice that a critical theme code breaking change on login was made.

Yet the official response given on the forum was look at the change log and compare difs, well as shown above the change log didn't list the change and there were a total of 164 changed files and templates and 3 removals in MyBB 1.8.16.

I did read all this before I updated looking for anything labeled as needing to be changed in the templates that would be forum breaking like that login issue but didn't see any major breaking changes like that listed.

Also telling people to compare diffs is quite hard before they update to changes that can break there site as the only diff compare method is once a site has been updated if you have a second copy of the default theme to cleanly compare diffs to as custom themes are all modified in 50+ files to the official ones as is, which isn't good to be seen as telling people to update to see whats changed and could break their sites.
Not everyone runs a secondly test forum, just like not everyone runs a second computer or a VM to test windows and other software updates.

I however have a secondary and as many domain test sites as I want.




This is something that should be addressed with documentation for the next version and before 2.0 that all theme and plugin changing critical bugs get notified in the release notes so users aren't caught out like they have been with the login.
You managed to mention max login attempt which was a rarer bug but not the login change which is something common to nearly every custom theme.



With 1.9 which will start becoming huge code changes to 2.0 a complete rewrite I hope MyBB can put in new procedures on QA and documentation of release to avoid simple mistakes as well as issues like this one with the login.

I am not trying to start a flame war or annoy anyone and I know as developers you may feel you owe no-one even any user or forum owner on here anything and that MyBB is already going out of its way to just provide updates and a release page as it is (this has been the vibe I have gotten on these forums from developers both main and plugins)

As a non profit forum company the goals of MyBB are to enable people to run there own forum as well as enable communication on the internet.

Taken from the front page.
Quote:Forum software everyone can love.
MyBB powers hundreds of thousands of engaging, powerful, and unique communities across the internet.

As enabling the community, so too should be releases and documentation be community focused as we are the ones impacted by changes here and need information in order to run our communities efficiently without issues from minor bugs and code changes that change how we run things.

I personally don't mind for major changes as long as we are informed what those changes are and how to deal with them.

With MyBB deciding to become new complete rewrite software as 2.0 so too should the ways and methodology be improved to enable more information and QA for users as a better stronger and more reliable system.

For info: I have run a few communities in the past on different board softwares.
I ran a Minecraft server with over 40, 3rd party plugins for two years as well as a forum and learned about the importance of changes with documentation and when even though we had the info right in their face people would constantly scream they weren't told what changed 'also as a means to troll me'.
I have a certificate 1 and 2 in business and worked as a Public Servant for a year with the Police department here.
I have a certificate 3 in I.T as well as 2-3 years working with disabled and underprivileged as a repair technician and kinda consultant liaison of info and procedures and improvement for the non profit Microsoft MAR refurbisher program places I was with.

So I know the importance of QA, documentation, also training, to the disabled and people that don't know a clue about using a PC the 'general moms and dads' as we called them that want to just browse the web, check email, play solitaire etc.
Reply
#2
Baseline: If it is potentially forum-breaking: make explicit and thorough in-our-face warnings that are prominent everywhere.

"Everywhere" meaning the release thread, the blog entry, the update page, the download page, github, maybe even IN the zip packages README.txt file.
Reply
#3
Your points are well taken. We don't deny there are mistakes at our part. It was a huge release 61 (actually 66, to be specific) bug fixes were not happened in a release in recent past - simply that was required even more dedicated testing.

Not defending but the fact is the active developers are counted as on date and focus is diverted to multiple development branches. We took time to release 1.8.16 but we did huge with it. Its too sad to feel as a person that we know we had put real efforts to make it better but somehow the testing phase was not adequately executed and our efforts failed to make our users happy.

During testing phase we required more people to be involved due to the lack of current team member availability but didn't get any support from the community. Our contributors stood outstanding and caught a number of issues which were instantly fixed. We even updated the community forums much prior to the release to use the mass active userbase who use the community forums on daily basis but daily required tools (like report fail, or simply take the login issue) were buggy, nobody noticed or reported. I appreciate your offer of the test board, but the fact is nobody even care unless it is his / her own site facing the bug. Still, it has already been discussed vastly and we are likely going to have an official test / demo setup up shortly.

All those reported issues were fixed within like 30 mins of identification, all of them. We are always with our users to provide support in every case, guide you step-by-step to keep your site problem free. We are already in process of a fixed release expected by next week. What has happened was not intentional and major reasons behind it is known to us.

Thanks for the heads up. Its an unfortunate event what was just happened and nobody is happy. But we assure we are with you to keep your site run smooth and ofcourse consider the documentation part to be more specific in future.

Sorry for all the inconvenience happened.
Reply
#4
I am very sorry and sad myself to hear that people don't seem to care.

I myself faced the same thing when running Minecraft and every other thing I tried to take a active roll in, I would join somewhere and eventually end up running the place or maintaining organisation due to my own high standards of polish and ways to properly do things while everyone else didn't care.
Its one reason I quit running the Minecraft server as well as the trolling and also a reason I left one I.T volunteering place as I was pretty much the only one that could do anything to a capable standard when others didn't care then told me off for trying to maintain that.

So I can relate in my own life.

In no way was my post above a way to take a stab it was more a reflection of what I have seen and read the last few releases in the most mature logical business minded I can be as this is how I try to live my life on most things.

I greatly appreciate the condolences on your behalf for the team to the community, there are good people out there that won't just defend and throw things back at the community (have also experienced this before)

With myBB 2,0 it looks like it could possibly be in a position to take on SMF PHPBB and even the likes of Xenforo as a free board and why I chose this over SMF, PHPBB etc plus the backend was very simple compared to them.

I would probably suggest currently you have team meetings about improving the outward community focus and interaction and engage here in the forums with the community asking for input on changes on how things could be done to improve this.

You could even make a temporary board dedicated for customer feedback on customer relations etc, do surveys etc.

Don't be afraid to ask for the community to tell you how you improve your outward focus and management systems its a invaluable tool.

Also as you mentioned you don't have the user base for internal testing I would recommend public testing versions with a warning that there will be bugs and issues and to use at risk.
Its a way to gain bug reports and release a version publicly like 1.9 was but with a beta tag that shouldn't be used on critical production boards.

This is a way to solve the bug issue for 1.8.17, release a public beta for a few weeks as a 1.8.17 RC, then fix up any issues for the stable see how that goes on a few versions and I can assure you things would improve.

I completely understand most people care only about their forums, me I'm trying to start a community but its hard when forums are no longer the 'thing'

With 1.9 and 2.0 things have a chance to really improve and it shouldn't just be the code, its also a chance for new procedures and a chance to work closer with the community.

Also are there any back end processes you can automate to help with the load?
I know the ReactOS project had issues with not having enough developers so they took some time out, automated and created new tools even diagnostic tools for a lot of there back end systems and procedures which has helped increase workflow as well as being invited to Googles summer of code and working on community focused and wanted features now like themes and certain applications the community wanted compatible etc to increase recognition.

Stuff like that will go a long way compiling a list of user wanted features and getting some of those prioritised as well as new back end tools and stuff.
So its something to consider with polishing up back end processes, being more open to public with the development process and working more with the community with feedback, surveys, feature requests etc.

First thing is to find out what we all need and want and would help to bring more people into into helping and contributing even bug reports on betas etc.
Microsoft opened up with windows insider and it was a huge success mby something a MyBB insider could work here with priority versions for testing to well known plugin and theme developers etc or anyone as long as people give feedback on them etc.

All things to take to the team and think about.
Reply
#5
I agree with the fact that there needs to be clearer wording as to what can potentially break a main function in your forum, in the blog. I.e the postkey situation.

Thanks for your feedback.
Plugin Count: I lost count.
Public Plugins are available here.
Official GitHub.
Please do not PM me for support unless asked to.
Reply
#6
(2018-07-06, 06:35 AM)effone Wrote: During testing phase we required more people to be involved due to the lack of current team member availability but didn't get any support from the community.

Again, any call for that must have went completely unnoticed by ma. OK, I have ADHD and sometimes don't see the things right in front of me, but maybe you could make such things a bit more visible? How about having a "sandbox" installation that is always available to the community where people can test and try and check the version to be released next?
Reply
#7
(2018-07-06, 07:05 AM)drguild Wrote: I am very sorry and sad myself to hear that people don't seem to care.

I myself faced the same thing when running Minecraft and every other thing I tried to take a active roll in, I would join somewhere and eventually end up running the place or maintaining organisation due to my own high standards of polish and ways to properly do things while everyone else didn't care.
Its one reason I quit running the Minecraft server as well as the trolling and also a reason I left one I.T volunteering place as I was pretty much the only one that could do anything to a capable standard when others didn't care then told me off for trying to maintain that.

So I can relate in my own life.

In no way was my post above a way to take a stab it was more a reflection of what I have seen and read the last few releases in the most mature logical business minded I can be as this is how I try to live my life on most things.

I greatly appreciate the condolences on your behalf for the team to the community, there are good people out there that won't just defend and throw things back at the community (have also experienced this before)

With myBB 2,0 it looks like it could possibly be in a position to take on SMF PHPBB and even the likes of Xenforo as a free board and why I chose this over SMF, PHPBB etc plus the backend was very simple compared to them.

I would probably suggest currently you have team meetings about improving the outward community focus and interaction and engage here in the forums with the community asking for input on changes on how things could be done to improve this.

You could even make a temporary board dedicated for customer feedback on customer relations etc, do surveys etc.

Don't be afraid to ask for the community to tell you how you improve your outward focus and management systems its a invaluable tool.

Also as you mentioned you don't have the user base for internal testing I would recommend public testing versions with a warning that there will be bugs and issues and to use at risk.
Its a way to gain bug reports  and release a version publicly  like 1.9 was but with a beta tag that shouldn't be used on critical production boards.

This is a way to solve the bug issue for 1.8.17, release a public beta for a few weeks as a 1.8.17 RC, then fix up any issues for the stable see how that goes on a few versions and I can assure you things would improve.

I completely understand most people care only about their forums, me I'm trying to start a community but its hard when forums are no longer the 'thing'

With 1.9 and 2.0 things have a chance to really improve and it shouldn't just be the code, its also a chance for new procedures and a chance to work closer with the community.

Also are there any back end processes you can automate to help with the load?
I know the ReactOS project had issues with not having enough developers so they took some time out, automated and created new tools even diagnostic tools for a lot of there back end systems and procedures which has helped increase workflow as well as being invited to Googles summer of code and working on community focused and wanted features now like themes and certain applications the community wanted compatible etc to increase recognition.

Stuff like that will go a long way compiling a list of user wanted features and getting some of those prioritised as well as new back end tools and stuff.
So its something to consider with polishing up back end processes, being more open to public with the development process and working more with the community with feedback, surveys, feature requests etc.

First thing is to find out what we all need and want and would help to bring more people into into helping and contributing even bug reports on betas etc.
Microsoft opened up with windows insider and it was a huge success mby something a MyBB insider could work here with priority versions for testing to well known plugin and theme developers etc or anyone as long as people give feedback on them etc.

All things to take to the team and think about.

In regards to people "not caring" about doing beta testing, the main reason is that a lot of people don't have the time and/or resources to do extensive beta testing.

As for people as myself who are running a forum based website, the majority of us are not coders, nor do we understand the detailed aspect of SQL databases. For whatever reason why we are running a forum based website should NOT ever be a hindrance in running one.

My reason for running my forum is different from DrGuild's reason for running one, but regardless of the reason, MyBB.com developers (be known as MyBB.com for the rest of this post) should NEVER assume that we are both alike in coding and SQL experience.

Yea, I can see some of you high-end coders telling me and others that we need to learn PHP and CSS. Why are you forcing us to learn this? Why are you hiding behind the rationale that because we don't have a certain academic background that we aren't worthy of running MyBB and shouldn't be running a website? Really? Really??

MyBB.com should never assume that everyone will understand the details of Diff files or understand things like modifying template files.
I have seen a number of threads/posts on MyBB.com where admins like myself are clueless and some get treated like we are a bunch of dummies. That's bad customer service!

When I started running my forum website about 19 months ago, I had ZERO experience in understanding a lot of the AdminCP stuff, but today, I am far more comfortable, but I took me that long to really be confident in making changes here and there.
A lot of the new admins are in a similar situation as I was when I started out. Please for the sake of the future of MyBB, don't treat those new admins like dummies. It's not cool and looks bad on MyBB in general.

In the situation regarding 'Authorization code mismatch. Are you accessing this function correctly?' issue, I could not believe some of the responses by MyBB.com that we admins "should have known better" and "should have read the Diff reports". Seriously? It was largely because of ONE line that was missing.
How would someone like myself know this? This is one of those times where MyBB.com could have taken the high road and made a thread or sticky a thread like DrGuild showing how to fix it, but give step-by-step instructions. Don't throw those fixes in and expect everyone to know what the hell you're talking about.

Getting off the 'soap box'.
I'm Serpius and You're Not    ¯\_(ツ)_/¯
LUXOR FORUMS
Reply
#8
(2018-07-06, 05:20 AM)drguild Wrote: The 6 Security vulnerabilities addressed doesn't list the change
The changes address the Login CSRF vulnerability; we've added a note to the Blog regarding login issues.

The Portal login form was indeed missed when modifying forms with security patches.

Quote:Also if It could have been held until 1.9 that would have been better than pushing out a theme breaking change when the whole theme system is being redone.
Currently all confirmed security issues are being addressed upon release with some publicly-known exceptions that are deferred with accepted risk due to limited resources (e.g. weak session system, password storage or the entire ACP section written insecurely - https://github.com/mybb/mybb/issues/3126) and expected to be addressed in future branches.

Quote:Also there is no way to click on the files in the changed files list to see just the changes to each file and template before updating.
Quote:Yet the official response given on the forum was look at the change log and compare difs, well as shown above the change log didn't list the change and there were a total of 164 changed files and templates and 3 removals in MyBB 1.8.16.
Quote:Also telling people to compare diffs is quite hard before they update to changes that can break there site as the only diff compare method is once a site has been updated if you have a second copy of the default theme to cleanly compare diffs to as custom themes are all modified in 50+ files to the official ones as is, which isn't good to be seen as telling people to update to see whats changed and could break their sites.
MyBB 2.0 was expected to be handled with greatly overhauled organization and development/release management - as certain aspects are dependent on the design of the whole codebase, some of them will have to be curbed.
With 1.8 the code still requires some changes to allow anyone download the latest development version from the repository and e.g. test upgrading; this would be more straightforward with database migrations (as opposed to a one-way script that can only be executed once) that would also allow for smaller, more frequent or even continuous updates. This would, however, require more attention when merging Pull Requests as all development branches would be expected to contain stable code.

MyBB users shouldn't be expected to have development skills to be able to run forums, but unfortunately there is a lot of loose ends to take care of when dealing with outdated code - especially in parallel to 1.9 development - taking a toll on security, design and overall experience (which we will aim to at least keep documented).

Users delaying updates is another problem and we might consider splitting security and feature/maintenance releases as vulnerability patches need to be applied on all boards as soon as possible.
devilshakerz.com/pgp (DF3A 34D9 A627 42E5 BC6A 6750 1F2F B8AA 28FF E1BC) ▪ keybase.io/devilshakerz
Reply
#9
I'll read more replies later add I'm on mobile.

I was thinking, it's been mentioned that you guys have tried to get more people testing testing for prereleases.
I didn't see anything around for this.
If you just posted on git etc not many people would see it.
It wasn't on the blog or main areas.

What I'm thinking of you use the inboard notice system for this.
That way all forum owners are notified that you need testing of a upcoming release.
Also they can see the main and prerelease versions on their own boards.

So instead of the forum notice being just your blog you can categories it into sections.
  • News
  • Updates
  • Blog
  • Latest highlights themes plugins etc

This way people can see directly in the forum feed area all the notices and cool things including new highlisted themes and plugins like wordpress running your blog does.

Updates: 1.8.17-prelelease, Testers Needed
We are hard at work on a new update but we want to ensure this release goes smoothly so we are encouraging everyone with a spare board to test and give us feedback on the forum area here (url)
You can download the prereleases version zip files and core it's changeover here (url)

This way everyone running their own board will see it in the admin area.
As your tester message you mentioned wasnt to me in a place the general owners frequent.
This solves that directly.

Also overhauling the board message and latest version system allows you to push more information directly to the users in their own environment.
This way you go to them instead of them coming to you, it's a win win.

Here is a quick mockup I did in chrome developer tools.
Note the mock url link on latest preview this is so when clicked it takes you directly to the changelog and notice on here about the prerelease where you can have a dedicated page for explaining the update and prerelease version etc.
Not too much code change would be required for this and shows a major improvement to the admin area to show the stuff I said above with prelease and more notices.
I tried to show that the announcement box can be turned in a tabbed view with different more filled out notices and messages giving you a bigger way to interact with the community right from the admin area.

[Image: oGrJUMG.png]
Reply
#10
(2018-07-07, 03:15 AM)drguild Wrote: I'll read more replies later add I'm on mobile.

I was thinking, it's been mentioned that you guys have tried to get more people testing testing for prereleases.
I didn't see anything around for this.
If you just posted on git etc not many people would see it.
It wasn't on the blog or main areas.

What I'm thinking of you use the inboard notice system for this.
That way all forum owners are notified that you need testing of a upcoming release.
Also they can see the main and prerelease versions on their own boards.

So instead of the forum notice being just your blog you can categories it into sections.
  • News
  • Updates
  • Blog
  • Latest highlights themes plugins etc

This way people can see directly in the forum feed area all the notices and cool things including new highlisted themes and plugins like wordpress running your blog does.

Updates: 1.8.17-prelelease, Testers Needed
We are hard at work on a new update but we want to ensure this release goes smoothly so we are encouraging everyone with a spare board to test and give us feedback on the forum area here (url)
You can download the prereleases version zip files and core it's changeover here (url)

This way everyone running their own board will see it in the admin area.
As your tester message you mentioned wasnt to me in a place the general owners frequent.
This solves that directly.

Also overhauling the board message and latest version system allows you to push more information directly to the users in their own environment.
This way you go to them instead of them coming to you, it's a win win.

Here is a quick mockup I did in chrome developer tools.
Note the mock url link on latest preview this is so when clicked it takes you directly to the changelog and notice on here about the prerelease where you can have a dedicated page for explaining the update and prerelease version etc.
Not too much code change would be required for this and shows a major improvement to the admin area to show the stuff I said above with prelease and more notices.
I tried to show that the announcement box can be turned in a tabbed view with different more filled out notices and messages giving you a bigger way to interact with the community right from the admin area.

This is a great idea to add that to the Admin main page.

I would like to add that the 'Check For Updates' tab page include the same information of pending or pre-release information so that us Admins are prepared ahead of time.
Even on that 'Check For Updates' page, step-by-step instructions can be added on so that everything is in front of the Admin.

Often, weeks or months go by with no version updates that us Admin don't pay much attention to the upcoming releases. We get busy with other aspects of our lives and often don't take the time to read up on the latest releases of MyBB.

I think adding that information to those 2 areas may cut down on "surprises".
I'm Serpius and You're Not    ¯\_(ツ)_/¯
LUXOR FORUMS
Reply


Forum Jump:


Users browsing this thread: 1 Guest(s)