MyBB Community Forums

Automated testing works well for decently organised codebases. Unfortunately, MyBB 1.x is not something that most people would call "well organised". I've tried various PHP linters and code analysis tools over the years and hit a great wall of false positives.

Relying on automated testing would be a bad call on our part, because automated testing doesn't catch everything.

Unit tests are great, but they only catch syntax errors and logical issues relating to the thing the test specifically tests. This requires developers to test for every possible issue, which is not really viable for big software solutions. Unit tests test logic, but don't test user interface or highlight user experience problems. Its also functionally difficult to have unit tests that test user interaction.

We regularly have people "report" SQL vulnerabilities based off automated tools, despite the fact most of those are just false positives.

Think of it like a taking your car for a service. Most modern cars have computers these days (engine management systems etc...) that can perform automated self testing and highlight issues. But just because the EMS says the engine and core systems are fine doesn't mean the mechanic should just sign the car off and say "yeah its done". They have to check things like tire tread, suspension alignment, and fluid levels manually.

Also MyBB is designed to run on many systems of varying power and performance. If we merged in some code that "looked alright" in regards to search performance or query performance on large boards that may turn out fine. Or it could detrimentally affect smaller boards. We wouldn't know without testing manually on test boards. Sure, you could set page loading time limits through some testing, but there's few systems that can also monitor server resources and integrate those measurements into testing.

2.0 will definitely use a lot more automated testing to test basic logic (like permissions) but UX changes pretty much have to be manually checked.

(2017-08-25, 12:47 PM)Tom K. Wrote: [ -> ]Relying on automated testing would be a bad call on our part, because automated testing doesn't catch everything.

Unit tests are great, but they only catch syntax errors and logical issues relating to the thing the test specifically tests. This requires developers to test for every possible issue, which is not really viable for big software solutions. Unit tests test logic, but don't test user interface or highlight user experience problems. Its also functionally difficult to have unit tests that test user interaction.

We regularly have people "report" SQL vulnerabilities based off automated tools, despite the fact most of those are just false positives.

Think of it like a taking your car for a service. Most modern cars have computers these days (engine management systems etc...) that can perform automated self testing and highlight issues. But just because the EMS says the engine and core systems are fine doesn't mean the mechanic should just sign the car off and say "yeah its done". They have to check things like tire tread, suspension alignment, and fluid levels manually.

Also MyBB is designed to run on many systems of varying power and performance. If we merged in some code that "looked alright" in regards to search performance or query performance on large boards that may turn out fine. Or it could detrimentally affect smaller boards. We wouldn't know without testing manually on test boards. Sure, you could set page loading time limits through some testing, but there's few systems that can also monitor server resources and integrate those measurements into testing.

2.0 will definitely use a lot more automated testing to test basic logic (like permissions) but UX changes pretty much have to be manually checked.

The SQL Vulns are real - they just aren't exploitable - you might not be able to dump a database but you can still cause an SQL Error to occur. Will MyBB 2.0 use Continuous Integration checking? Also if MyBB 1.8 has a messy codebase let's start a project to fix that

(2017-08-25, 06:29 PM)Lunorian Wrote: [ -> ]Also if MyBB 1.8 has a messy codebase let's start a project to fix that

It's already there – MyBB 2.0 is meant to fix it.

(2017-08-25, 06:29 PM)Lunorian Wrote: [ -> ]

(2017-08-25, 12:47 PM)Tom K. Wrote: [ -> ]Relying on automated testing would be a bad call on our part, because automated testing doesn't catch everything.

Unit tests are great, but they only catch syntax errors and logical issues relating to the thing the test specifically tests. This requires developers to test for every possible issue, which is not really viable for big software solutions. Unit tests test logic, but don't test user interface or highlight user experience problems. Its also functionally difficult to have unit tests that test user interaction.

We regularly have people "report" SQL vulnerabilities based off automated tools, despite the fact most of those are just false positives.

Think of it like a taking your car for a service. Most modern cars have computers these days (engine management systems etc...) that can perform automated self testing and highlight issues. But just because the EMS says the engine and core systems are fine doesn't mean the mechanic should just sign the car off and say "yeah its done". They have to check things like tire tread, suspension alignment, and fluid levels manually.

Also MyBB is designed to run on many systems of varying power and performance. If we merged in some code that "looked alright" in regards to search performance or query performance on large boards that may turn out fine. Or it could detrimentally affect smaller boards. We wouldn't know without testing manually on test boards. Sure, you could set page loading time limits through some testing, but there's few systems that can also monitor server resources and integrate those measurements into testing.

2.0 will definitely use a lot more automated testing to test basic logic (like permissions) but UX changes pretty much have to be manually checked.

The SQL Vulns are real - they just aren't exploitable - you might not be able to dump a database but you can still cause an SQL Error to occur. Will MyBB 2.0 use Continuous Integration checking? Also if MyBB 1.8 has a messy codebase let's start a project to fix that

Yes, 2.0 currently uses Travis CI.

(2017-08-25, 07:06 PM)Shade Wrote: [ -> ]

(2017-08-25, 06:29 PM)Lunorian Wrote: [ -> ]Also if MyBB 1.8 has a messy codebase let's start a project to fix that

It's already there – MyBB 2.0 is meant to fix it.

It's waaay too late to rewrite 1.8. MyBB would probably collapse before that's done.

The only way forward is with 2.0.

(2017-08-26, 07:30 AM)Azah Wrote: [ -> ]It's waaay too late to rewrite 1.8. MyBB would probably collapse before that's done.

The only way forward is with 2.0.

You think MyBB will collapse before they fix their existing software yet they'll stay around long enough to write new software?

I'm closing this because its going nowhere. We do test releases, and we're not going to suddenly stop doing that just to push out updates faster. While it might save time in the short term, but long term its just going to lead to many bugs not being caught. Anyone with any real experience in development would realise this.

Because previous thread was closed by Tom from obvious reasons (having nothing constructive to say), I am creating this one, and will create as long as I like.

Here (post #14) Lumorian is right :: MyBB does not have devs that are fluent in PHP. Thats the main reason why there are tests being done and why does it take soo ling for an update to be released.
New PRs must be tested before being merged into repo? [snip] Its against free contribution idea. Besides it doesnt make sense. If one writes code thats poor quality and sends PR, accept it.
Learn to accept the fact that everyone can contribute freely (=without any restriction as of code quality).


#19:: Again, if you are fluent in PHP, its a breeze.....
#21:if MyBB's code is not well organised, than make it well organised. Suspend releases till its fully rewritten. Plain simple.