this makes sense, in 2018-2022, I would get tons of emails from recruiters at meta, doordash, snap, stripe etc now I barely see any (maybe they've given up )
My friends who are still at Google also say that most job postings will end up going to someone internally - in fact people say they don't do that many external interviews anymore.
Finally the interview cycle seems to take a lot longer than I remember with quite a few added rounds.
There were some viral threads going around for a bit that were focused on this word. It never made sense to me - presumably an LLM that outputs the word delve does so because it learned it from the data it was trained on, meaning delve was already a common word.
I personally have used the word for many years, and have seen it extensively pre-LLM. To me, it’s only a red flag when someone starts producing content that doesn’t match their prior history of writing with a sudden explosion of vocabulary.
For coding it is still 10x worse than gpt4. I asked it to write a simple database sync function and it gives me tons of pseudocode like `//sync object with best practices`. When I ask it to give me real code it forgets tons of key aspects.
Because they're ultimately training data simulators and not actually brilliant aritifical programmers, we can expect Microsoft-affiliated models like ChatGPT4 and beyond to have much stronger value for coding because they have unmediated access to GitHub content.
So it's most useful to look at other capabilities and opportunities when evaluating LLM's with a different heritage.
Not to say we shouldn't evaluate this one for coding or report our evaluations, but we shouldn't be surprised that it's not leading the pack on that particular use case.
Github full (public) scrape is available to anyone. GPT-4 was trained before Microsoft deal so I don't think it is because of Github access. And GPT-4 is significantly better in everything compared to second best model for that field, not just coding.
Someone doesn't get good at programming with low quality learning sources. Also, a poor comparison because models are not people - might as well complain about how NPCs in games behave because they fail at problems real people can solve.
We are both substrate that has been aggressively optimized for a task with a lot of side benefits. "NPC"s are not optimized at all, they are coded using symbolic rules/deterministic behavior.
Zero chance private github repos make it into openai training data, can you imagine the shitshow if GPT-4 started regurgitating your org's internal codebase?
Agreed, but I do find gpt4 has been increasing the amount of pseudo code recently. I think they are a/b testing me. I find myself asking if how much energy it wasted giving me replies that I then have to tell it to fix.. Which is of course a silly thing to do, but maybe someone at oAI is listening?
That can't be, because I can ask it a simple question that an answer is maybe 1 sentence, and it repeats the question then provides a whole novel. So ton of tokens.
Yeah but to be honest been a pain last days to get gpt 4 to write full pieces of code for more the 10-15 lines. Have to re-ask many times and at some point it forgets my initial specifications.
Earlier in the year I had ChatGPT 4 write a large, complicated C program. It did so remarkably well, and most of the code worked without further tweaking.
Today I have the same experience. The thing fills in placeholder comments to skip over more difficult regions of the code, and routinely forgets what we were doing.
Aside all the recent OpenAI drama, I've been displeased as a paying customer that their products routinely make their debut at a much higher level of performance than when they've been in production for a while.
One would expect the opposite unless they're doing a bad job planning capacity. I'm not diminishing the difficulty of what they're doing; nevertheless, from a product perspective this is being handled poorly.
Definitely degraded. I recommend being more specific in your prompting. Also if you have threads with a ton of content, they will get slow as molasses. It sucks but giving them a fresh context each day is helpful. I create text expanders for common prompts / resetting context.
eg:
Write clean {your_language} code. Include {whatever_you_use} conventions to make the code readable. Do not reply until you have thought out how to implement all of this from a code-writing perspective. Do not include `/..../` or any filler commentary implying that further functionality needs to be written. Be decisive and create code that can run, instead of writing placeholders. Don't be afraid to write hundreds of lines of code. Include file names. Do not reply unless it's a full-fledged production ready code file.
These models are black boxes with unlabeled knobs. A change that makes things better for one user might make things worse for another user. It is not necessarily the case that just because it got worse for you that it got worse on average.
Also, the only way for OpenAI to really know if a model is an improvement or not is to test it out on some human guinea pigs.
My understanding is they reduced the number of ensembles feeding gpt4 so they could support more customers. I want to say they cut it from 16 to 8. Take that with a grain of salt, that comes through the rumor telephone.
Are you prompting it with instructions about how it should behave at the start of a chat, or just using the defaults? You can get better results by starting a chat with "you are an expert X developer, with experience in xyz and write full and complete programs" and tweak as needed.
Yep, I'm still able to contort prompts to achieve something usable; however, I didn't have to do that at the beginning, and I'd rather pay $100/mo to not have to do so now.
OpenAI just had to pause signups after demo day because of capacity issues. They also switched to making users pay in advance for usage instead of billing them after.
They aren't switching anything with payments. Bad rumor amplified by social contagion and a 100K:1 ratio of people talking about it to people building with it.
Im not really sure what chatgpt+ is serving me. There was a moment it was suddenly blazing fast, that was around the time turbo came out. Off late, it's been either super slow or super fast randomly.
Try using the playground, with a more code specific system prompt, or even put key points/the whole thing into the system prompt. I see better performance, compared to the web.
This has exactly been my experience for at least the last 3 months. At this point, I am thinking if paying that 20 bucks is even worth anymore which is a shame because when gpt-4 first came out, it was remembering everything in a long conversation and self-correcting itself based on modifications.
Since I do not use it every day, I only pay for API access directly and it costs me a fraction of that. You can trivially make your own ChatGPT frontend (and from what people write you could make GPT write most of the code, although it's never been my experience).
definitely noticed it being "lazy" in the sense it will give the outline for code and then literally put in comments telling me to fill out the rest, basically pseudocode. Have to assume they are trying to save on token output to reduce resources used when they can get away with it
Even when I literally ask it for code it will often not give me code and will give me a high level overview or pseudocode until I ask it again for actual code.
It's pretty funny that my second message is often "that doesn't look like any programming language I recognize. I tried running it in Python and got lots of errors".
"My apologies, that message was an explanation of how to solve your problem, not code. I'll provide a concrete example in Python."
I had one chat with ChatGPT 3.5 where it would tell me the correct options (switches) to a command, and then a couple weeks later it is telling me this (in the same chat FWIW):
> As of my last knowledge update in September 2021, the XY framework did not have a --abc or --bca option in its default project generator.
Except: you can feed it an entire programming language manual, all the docs for all the modules you want to use, and _then_ it's stunningly good, whipping chatgpt4 that same 10x.
I gather the pricing is $8 for a million input tokens [1] so if your language's manual is the size of a typical paperback novel, that'd be about $0.8 per question. And presumably you get to pay that if you ask any follow-up questions too.
Sounds like a kinda expensive way of doing things, to me.
Claude? No, have requested access many times but radio silence.
OpenAI? I use ChatGPT A LOT for coding as some mixture of pair programmer and boilerplate, works generally well for me. On the API side use it heavily for other work and its more directed and have a very high acceptance rate.
Can you just tell it to focus on a particular language and have it go find the manuals?
If it is so easy to add manuals, maybe they should just make options to do that for you.
Am I only one that thinks that Claude 2 is not bad for programming questions? I do not think it is best one for programming questions but I do not think that it is bad too. I have received multiple times very good response from Claude 2 on Python and SQL.
I find all of them, gpt4 or not, just suck, plain and simple. They are only good for only the most trivial stuff, but any time the complexity rises even a little bit they all start hallucinate wildly and it becomes very clear they're nothing more than just word salad generators.
I have built large scale distributed gpu (96gpus per job) dnn systems and worked on very advanced code bases.
GPT4 massively sped up my ability to create this.
It is a tool and it takes a lot of time to master it. Took me around 3-6 months of every day use to actually figure out how. You need to go back and try to learn it properly, it's easily 3-5x my work output.
In case this is useful - I had end stage arthritis in one of my ankles and ended up getting a bipolar ankle allograft in San Diego at Scripps. I barely think about it nowadays except that sometimes after a long hike or strenuous workout it will feel a bit stuff. Considering I could barely walk before it was a night and day difference.
You essentially get a completely new joint surface from a donor and have to be in crutches for 3 months but it was 100% worth it for me. I refused to do fusion and avoided any doctor that said nothing could be done.
I can confirm. Just moved from Zurich to San Francisco. Pretty much everything except rent is cheaper in San Francisco (food can be up to half price of what I paid in Zurich) but rent being such a major expense, it ends up being similar. Salaries are a bit higher in Switzerland though so one would save more on average there.
They democratically elected to basically jump off a cliff
Scare-mongering. If you have to be in the EU to be able to trade, yes, to survive, then it is time to destroy the EU once and for all. No single entity should be able to terrorize the citizens of a nation state like that.
Straw man - no one is saying we won't survive but you would have to be monumentally dim to not perceive that what has just happened, and what will continue to happen in Britain as a result of this decision will be orders of magnitude worse than the alternative - at least in the short to medium term. I can't predict the long term future and it may well be that the UK somehow manages to come out of this with more than the south west still part of it but the path there will involve a lot of suffering.
He said jumping off the cliff. That is literally just saying the Brits won't survive. Besides, England will be able to make a trade deal with China (something the EU doesn't manage to do). Medium- to longterm, the benefits for the British people will be vast - and I'm not even starting to argue about political independence.
Jumping off a cliff != certain death, but I guess thats splitting hairs. You seem very confident in the face of expert opinion and evidence in front of you. Does the fact that the people who have masterminded this spectacle seem to have no plan for what happens next worry you in the slightest?
I notice you use England, possibly a mistake but it seems this is the underlying attitude of the Leave vote - England is better off by itself. Which strikes me as frankly hilarious given how much we have to offer the world besides grumpiness, self deprecation and financial services...
Well, the plan for what happens next depends in a large part on negotiations with the EU - who doesn't seem to care about negotiating if there is no clear wakeup call and time limit (like 2 years after taking article 50). The vote was a first step and clear signal that the British people are unhappy with how the EU treats them.
I meant to say Britain so yes it was an error. The tensions between England and Scotland (and to a lesser degree Gibraltar) are unfortunate, especially that the leave vote increased them again. Same with northern Ireland tensions. I have no opinion on whether or not they are supposed to stay together since this seems to be a problem rooted deeply in culture that I might not be able to fully comprehend.
> No single entity should be able to terrorize the citizens of a nation state like that.
Talk about hyperbole. Plenty of countries like Switzerland or Norway don't feel terrorised by the EU, instead exist peacefully alongside it. Plenty more want to join voluntarily.
Another fact that seems completely ignored- you can't put a price on deliberately creating links so that European powers don't tear each other to pieces, just like they have done for thousands of years. The EU deserves credit for minimising the likelihood of devastating war, which has happened for much of recorded history in Europe.
European wars, particularly the nastier ones are born out of hegemony and counter-hegemony. The EU was bound to be an hegemonic construct dominated by the country that could dominate it - which basically means France, Germany or Great Britain. Turned out to be Germany.
The utopia of a united Europe is what caused all of the devastating wars - whether it was under the guise of a Spanish empire, a Napoleonic empire or a German reich. Whether it is done by military, political or economic power, it always backfires.
We are different people. What we have in common is mostly Christian religion and Christian morals. Or rather the purified, distilled secularised version of it called "humanism" now.
But some visionary always comes with this radical new idea (actually a thousand years old) that we are somehow all the same and should all be in some sort of union. And it always ends up in war.
Apart from the campaigners on each side of the referendum.
Basically the whole thing descended into "FUCKING IMMIGRANTS TAKING YOUR JOBS, THATS WHY YOU DON'T HAVE PUBLIC SERVICES"
and on the other "YOU WON'T HAVE A JOB IF WE LOOSE THE EU. YOU WILL DIE OF AIDS"
Sadly people have been conditioned by various parts of the media to just see immigration as the reason for public service cuts. Not stopping to ask them selves, who makes the decision to cut the funding...
And this is entirely the fault of the political class, who wanted to mask cutting public services to pay for tax cuts for the middle class.
And now we have this right fucking mess, which will take many years to un pick, and I suspect will lead back exactly where we were before, just paying more, and having less influence.
Tbh I see the vote as a vote for democracy and self-determination. The immigrant question is secondary. The EU applies enormous political pressure all across the European continent and it's getting really uncomfortable, especially if the vast majority votes for something that is legally binding and your government can't act on it because of EU pressure (which happened here).
You see it as that, because I suspect you've taken a more than casual interest.
The concept of sovereignty is secondary in most cases. A lot of the votes "up north" were based on the assumption that public services are shit because of immigration.
This is very much an American thing to hate public transport though. Places in Europe, especially Switzerland for example, have much better infrastructure and taking a car can be a pain compared to a train which is faster and a lot more comfortable.
My friends who are still at Google also say that most job postings will end up going to someone internally - in fact people say they don't do that many external interviews anymore.
Finally the interview cycle seems to take a lot longer than I remember with quite a few added rounds.
reply