| T O P I C R E V I E W |
| foxmuldr |
Posted - Jan 16 2024 : 11:04:54 AM
I wanted to share my experience with Tabnine AI. I installed it back in November 2023 I believe, and have used it through December and into January. It's similar to github copilot in that it offers inline suggestions as you're coding. Some are lengthy, some are similar to auto-completion.
In my experience, it's about 30% effective in my coding. About 30% of the time if offers good suggestions, and of that 30% about 5% of the time it offers truly excellent suggestions. However, about 65% of the time it offers incorrect suggestions, and about 5% of the time it offers truly astounding and ridiculous suggestions. I had one today which prompted me to contact Tabnine and cancel my subscription.
I recognize that all of this technology is in its infancy, and I don't blame Tabnine. I think what I saw was a fluke as this is the first time in weeks of coding I've seen such a thing. Nonetheless... AI needs to be monitored.
Today, I was coding a service and I added this comment and pressed the Enter key:
// Set its child windows
Tabnine suggested this sequence (all on one line, literally out to column 677 in the editor):
SetWindowLongPtr(GetDlgItem(GetDlgItem(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(GetParent(
GetParent(GetParent(GetParent
I contacted Tabnine and asked to cancel my subscription and get a refund, and they promptly responded and issued a refund. I was already going to cancel Tabnine because although it does offer good suggestions at times, it's not worth the price for the suggestions it gives imho.
Since I cancelled the service and uninstalled the extension, I've gone back to just using auto-complete with VAX's suggestions, and those provided for by Visual Studio itself.
Except for that 5% portion of the 30% of suggestions, which is the portion related to it figuring out the name of a new function that's required based on context, I am finding my development is actually much faster merely using auto-code complete. It's primarily because I don't have to stop my typing and train of thought to read and interpret what's being suggested, but instead I already know what I'm looking for and just have to compare what I'm seeing with what I'm thinking for a simple go/no-go acknowledgement.
To be honest it's like a breath of fresh air to be back to normal development again only using auto-complete. I only use this form in my personal development, but now to be back at my regular job as well ... it's just fantastic.
I wanted to share that experience with you, VAX people, because I love the VAX tool. I've used it since 2006 and I consider it to be one of my greatest developer assets.
If you could ever add an intelligent module that would figure out the proper name for things (based on my project's naming conventions), which is somethin Tabnine was actually quite excellent at, then I would really appreciate that add-in.
For example, I have several functions named things like iiFile_open(), iiFile_read(), iiFile_close(), etc. The actual names are much more complicated, but that's the gist.
Today I added a comment that said "// Seek to the end of file" and pressed enter, and Tabnine suggested "iiFile_seekEnd(fh, " and that was a fantastic suggestion. I didn't have an iFile_seekEnd() written at this point, but it as able to determine what I was trying to do and constructed a properly named function for me to create.
I'm pretty sure the iFile*() and iiFile*() naming conventions I use are non-standard apart from my code. The i functions are internal, meaning they don't face an external/visible API, and the ii functions are internal to internal functions, which means they don't have to do parameter checking as they've already been vetted.
Tabnine was able to suggest names like that regularly, which was truly a marvelous feature. It even came up with some I wouldn't have thought of (like iiFile_seekEnd() as I would've just had an iiFile_seek() function and passed in the SEEK_SET, SEEK_CUR, SEEK_END constants). But it followed my naming convention pattern and it opened up my eyes to getting rid of that extra parameter used everywhere. I've used several like that.
-----
In my opinion, AI will only be beneficial if it can train to the individual user's method of writing code, learning from their patterns, and adding in suggestions the individual coder themselves might make given enough time to type it in.
Blindly trained AIs can be useful for some types of code, but they really need that custom honing to the individual to be truly useful.
If VAX ever decides to go the way of AI, I would offer that as a cue or suggestion to make your product most beneficial.
--
Rick C. Hodgin
|
| 1 L A T E S T R E P L I E S (Newest First) |
| feline |
Posted - Jan 17 2024 : 06:53:45 AM
An interesting experience.
I suspect I am overly suspicious of the claims of AI being the answer to everything, but having worked at support for a while now, some problems are really easy, while some are really hard, depending on very system specific combinations of programs, settings and code. Things that a general database of knowledge isn't likely to know anything about.
Plus years of finding that the "answers" and "solutions" posted online to technical problems don't always work, or even compile, sigh. There is a lot of good information out there, but sometimes the devil really is in the details. |
|
|
